Players get teleported to different server?

I'm using TeleportService:ReserveServer and TeleportService:TeleportToPrivateServer but for some reason (sometimes) not all players would teleport to that specific server and they would instead be alone in a different server. Is there a fix for this?

This is the script I'm using:

local TeleportService = game:GetService("TeleportService")
local TouchBrick = script.Parent
local placeId = 123
local Party = {}
TouchBrick.Touched:Connect(function(touch)
if touch.Parent:FindFirstChildOfClass("Humanoid") then
local Character = touch.Parent
local Player = game.Players:GetPlayerFromCharacter(Character)
for _, OtherPlayer in next,Party do
if OtherPlayer == Player or not Player:IsA("Player") then
return
end
end
table.insert(Party,Player)
end
end)
while true do
local reservedServer = TeleportService:ReserveServer(placeId)
local plrs = Party
if #plrs>0 then
TeleportService:TeleportToPrivateServer(placeId, reservedServer, plrs)
end
end