When a player joins a sound is put in their PlayerGui, but how do I get rid if it?

Looking at your code, I think it might be line 12; You are using the 'remove' method. As stated on the WIKI, ':remove()' only reverts the Object's Parent to nil, also, you are not checking if 'Sound' is infact in the PlayerGui, it is just executing that line without checking if it exists or not, thus the Script may break, same with lines 5, and 8.

This can be fixed, let's instead of using remove, let's use the Destroy method, and fix up your code;

local plr = game.Players.LocalPlayer --Judging by your code, this must be in a 'LocalScript'
repeat wait() until plr.Character --This will wait until the Player's Character has spawned, or is existant

function onButton1Down()
plr.Character:MoveTo(Vector3.new(-99.6, 19.5, 23.5))--Changed this from 'CFrame' to 'MoveTo'; what this will basically do is, revert the Character's CFrame to the 'Vector3''s exact coordinates, or, will, as you say, 'Teleport' the Player
repeat wait() until game.Workspace.CurrentCamera --This will wait until 'CurrentCamera' is existant; This will get the Player's Camera; This can only be called 'individually' by a 'LocalScript', I'm not too sure about Client-sided 'Script''s
local Ncam = game.Workspace.CurrentCamera --This will specify the 'CurrentCamera'
Ncam.CameraSubject = plr.Character:WaitForChild("Humanoid") --WaitForChild is used to, well, wait for a Child matching the Name until it has loaded/spawned
Ncam.CameraType = "Custom"
wait(0.2)
if plr:FindFirstChild("Backpack") and plr.Backpack:FindFirstChild("CamFix") then --Check to see if 'Backpack', and 'CamFix' exist
plr.Backpack.CamFix.Disabled = true --This will disable the 'CamFix' script
plr.Backpack.CamFix:Destroy() --Will destroy the script after disabling
end --This ends the code block for the 'if' statement
script.Parent.Parent.Parent:Destroy() --As you did before, but only I changed it to 'Destroy'; This will Destroy the Script's Parent's Parent.
if plr:FindFristChild("PlayerGui") and plr.PlayerGui:FindFirstChild("Sound") then --Check to see if 'PlayerGui', and 'Sound' exist
plr.PlayerGui.Sound:Stop() --The 'Stop' method is used to Stop a playing Audio/Sound
plr.PlayerGui.Sound:Destroy() --'Destroy' will destroy the Sound
end --This ends the code block for the 'if' statement
end --This ends the code block for the 'function'

script.Parent.MouseButton1Down:connect(onButton1Down) --Connects the event to the function; whenever the TextButton/ImageButton is clicked, it will fire the function 'onButton1Down'

Ah, doesn't your code look much better now? :)

As you might have seen in the Code, I have added if then end statements, if then end statements, basically check the condition of something, like it is checking for CamFix and Sound in the code, you may have also noticed I changed your Teleport, yes, I have change it to make it a bit more efficient. :)

The problem of how it could of kept Repeating is because you might have set PlayOnRemove to true, else you might have played it Twice at the same time when executing the code, otherwise I don't know.

Hope this helped!