Is it possible to not spawn the player's character until they press a GUI button?

What you want to use is the CharacterAutoLoads property and LoadCharacter method. Setting the CharacterAutoLoads property of the Players object to false will make it so that the character won't load until the LoadCharacter method is called on the Player. What you could do is set the CharacterAutoLoads property to false, and when the GUI button is clicked, the player's character is loaded, like so:

local btn = script.Parent
local plr  = btn.Parent.Parent.Parent.Parent --If the button is the child of a frame, which is a child of screengui, which is in turn a child of the playergui, which is a child of the player, this would index the player.
game.Players.CharacterAutoLoads = false

function onClicked()
    plr:LoadCharacter()
end

btn.MouseButton1Click:connect(onClicked)

Keep in mind that you should only use CharacterAutoLoads and LoadCharacter in server scripts.

If this helped you out, remember to accept the answer, and don't be afraid to ask questions :D