.touched event disconnection when a player stops holding E?

Can someone help me figure out how to get remote events done, I'm VERY new to client -- > server communications and I'm trying to make a simple hold to punch script

-- Local script
local UserInputService = game:GetService('UserInputService')
local ReplicatedStorage = game:GetService('ReplicatedStorage')
local Player = game.Players.LocalPlayer
local Character = game.Workspace:WaitForChild(Player.Name)
local Humanoid = Character:FindFirstChildOfClass('Humanoid')
local Punch = Instance.new('Animation')
Punch.AnimationId = 'rbxassetid://5224801450'
PunchTrack = Humanoid:LoadAnimation(Punch)

local punching = false

UserInputService.InputBegan:Connect(function(Input, Typing)
    if Typing then return end
	    if Input.KeyCode == Enum.KeyCode.E and punching == false then
		punching = true
		PunchTrack:Play()
		game.ReplicatedStorage.Punch:FireServer(punching)
end
end)

UserInputService.InputEnded:Connect(function(Input, Typing)
    if Typing then return end
	    if Input.KeyCode == Enum.KeyCode.E and punching == true then
		punching = false
		PunchTrack:Stop()
end
end)

-- Server script in ServerScriptService
game.ReplicatedStorage.Punch.OnServerEvent:Connect(function(player, punching)
	player.Character["Left Arm"].Touched:Connect(function(hitPart)
	if punching == true and hitPart.Parent:FindFirstChild('Humanoid') then
			local HumanoidToDamage = hitPart.Parent:FindFirstChild('Humanoid')
			HumanoidToDamage:TakeDamage(.2)
		end
		end)
end)

My problem is that when the player STOPS holding E the damage keeps going, how could I fix this? Feel free to DM me, they are open and would help me understand my discord is morgans bby#6569

When input ends all you do is stop the animation i don't anywhere that you communicate to the server to stop firing the event or to disconnect the touch event, maybe i'm missing something tho