How do I remove a spawned part when the local player has died/disconnected?

Clarification,

Functions and events can be hidden, and called when the script deems ready. This means we basically CAN do what you suggested. As K3du53 said, "the Died event is called on the humanoid".

Your main questions seems to be, "Where do I put the code?". This is rather simple. Because nothing will run after the first while loop, we should place the anonymous function above the while loop.

local Player = game.Players.LocalPlayer

Player.CharacterAdded:connect(function(PlayerBody)  

    BasicOrb = Instance.new("Part", game.Workspace)
    BasicOrb.Name = Player.Name.."'s_BasicOrb"
    BasicOrb.Size = Vector3.new(2,2,2)
    BasicOrb.Anchored = true
    BasicOrb.Shape = "Ball"
    BasicOrb.Material = "Neon"
    BasicOrb.BrickColor = BrickColor.new("Really red")

    PlayerBody:WaitForChild("Humanoid").Died:connect(function()
        BasicOrb:Destroy()
    end)

    while wait() do
        BasicOrb.CFrame = PlayerBody.Head.CFrame:toWorldSpace(CFrame.new(0,10,0))
    end
end)

Never use the Remove Function. This function has been deprecated. Use Destroy.

Precautions,

When the character dies, the Orb gets Destroyed. This will error in the while loop when the script tries to set the CFrame of a nil value. To fix this, I would check in the while loop if the Orb exists before setting it's CFrame will a basic conditional statement.

local Player = game.Players.LocalPlayer

Player.CharacterAdded:connect(function(PlayerBody)  

    BasicOrb = Instance.new("Part", game.Workspace)
    BasicOrb.Name = Player.Name.."'s_BasicOrb"
    BasicOrb.Size = Vector3.new(2,2,2)
    BasicOrb.Anchored = true
    BasicOrb.Shape = "Ball"
    BasicOrb.Material = "Neon"
    BasicOrb.BrickColor = BrickColor.new("Really red")

    PlayerBody:WaitForChild("Humanoid").Died:connect(function()
        BasicOrb:Destroy()
    end)

    while wait() do
        if BasicOrb then
            BasicOrb.CFrame = PlayerBody.Head.CFrame:toWorldSpace(CFrame.new(0,10,0))   
        else
            break;
        end
    end
end)

Break will "break out" of a loop.

Possible Future Errors

I understand you're using Local Player, and this should mean you're using a LocalScript. You should put this Local Script in ReplicatedFirst, StarterPack, StarterPlayerScripts, or some other replicated service that Local Scripts can be run in. However, depending on if you have FE, "Filtering Enabled", Enabled, you might want to make this system Server-Sided so other people can see the orbs. I wouldn't discourage using FE on a new place, but converting an old place to FE will almost certainly cause you problems. In the future however, make sure you understand when it's better to have things Client Sided Vs. Server Sided.

Good Luck!

It would be awesome if you accepted an answer. This answer doesn't have to be mine, but the one that helped you the most or solved your problem the best. Thank you.

Noting, if you were to turn FE on, the orb would only show for the player who owns the orb.

The function,

Player.Died:connect(function()
    BasicOrb:Remove()
end)

should work because BasicOrb is referenced earlier in the script. (but with some modifications)

First, the Died event is called on the humanoid, so that would be replaced with:

Player.Character.Humanoid.Died:connect(function()
    BasicOrb:Remove()
end)

Next, so that the orb doesn't stay hidden in nil (I'm not sure how the garbage collector works), we destroy reference to the orb, or call destroy on it.

Player.Character.Humanoid.Died:connect(function()
    BasicOrb:Remove()
    BasicOrb = nil
end)

OR

Player.Character.Humanoid.Died:connect(function()
    BasicOrb:Destroy()
end)

(I'm really bad at closing sentences. Good luck?!)