How do I make sure the player is on a specified brick when he clicks the button?

I am trying to make a timer for an obby. I want the player to be at the start of the obby when he clicks the button on his screen to start the timer. I tried doing this with Touched and TouchEnded events but they don't work properly. I will idle on the starting brick and Roblox will say I'm not touching the brick when I am clearly touching the brick. The way the system is set up if Roblox says I'm not on the brick then I can't click the start timer button. Is there any way possible I could do what I want without using Touch and TouchEnded events because the are not reliable?

Client Code

--Loads event which will be needed to start the timer
local startTimerEvent = game.ReplicatedStorage.StartTimerEvent
local player = game.Players.LocalPlayer
--Starting place of the obby
local startBrick = game.Workspace.StartObby
--Is the player touching the brick?
local touchedStart = false

startBrick.Touched:Connect(function(hit)
    --Checks to make sure the one who is touching the brick is the same player
    --and not some other touching the brick
    if game.Players:GetPlayerFromCharacter(hit.Parent) == player then
        touchedStart = true
        print("Touched Began")
    end
end)

startBrick.TouchEnded:Connect(function(hit)
    if game.Players:GetPlayerFromCharacter(hit.Parent) == player then
        touchedStart = false
        print("Touch Ended")
    end
end)

script.Parent.MouseButton1Click:Connect(function()
    if touchedStart then --If I am at the start then execute code
        print("Fired")
        --This next code tells the server to start the timer and stops the timer when
        --the player is dead.
        startTimerEvent:FireServer(player)
    end
end)