Why isn't my player list gui removing players when they leave the game?

I am working on a player list that makes a button for each player so that admins can do stuff like kick them in secret without them knowing. However, it does add a player when they join, but it doesn't remove them when they leave. Here is the script

Client:

onPlayer = game.ReplicatedStorage.OnPlayer
x = 0
y = 0.178
frame = game.Players.LocalPlayer:WaitForChild("PlayerGui"):WaitForChild("AskAdmin").Players
children = frame.Buttons:GetChildren()
buttonSize = UDim2.new(1, 0, 0.178, 0)

local function button(name, xPos, yPos, parent, textColor, backgroundColor)
	local button = Instance.new("TextButton")
	button.Parent = parent
	button.Text = name
	button.Position = UDim2.new(xPos, 0, yPos, 0)
	button.TextScaled = true
	button.Name = name
	button.Size = buttonSize
	button.BackgroundColor3 = backgroundColor
end

local function clearList()
	for i = 1, #children do
		children[i]:Destroy()
	end
end

local function createList()
	x = 0
	y = 0.178
	frame.CanvasSize = UDim2.new(0, 0, 0, 0)
	local players = game.Players:GetPlayers()
	
	for i = 1, #players do
		button(players[i].Name, x, y, script.Parent.Buttons, Color3.fromRGB(0, 0, 0), Color3.fromRGB(255, 255, 255))
		y = y + 0.178
		frame.CanvasSize = UDim2.new(0, 0, y, 0)
	end
	print("New list created")
end

onPlayer.OnClientEvent:Connect(function(player)
	createList()
end)

Server:

onPlayer = game.ReplicatedStorage.OnPlayer

game.Players.PlayerAdded:Connect(function(player) 
	onPlayer:FireAllClients() 
end)

game.Players.PlayerRemoving:Connect(function(player)
	onPlayer:FireAllClients()
end)

What I am basically doing is im using a remote event to let the client know when the player list will change. And it does add a player to the playerlist like I said earlier, but it doesn't remove them, despite it letting the client know that a player was removed. Basically, when the remote event is fired, the list will clear and then redo the player list now that its changed. What am I doing wrong?