A problem with a touched event (punch script)?

Alright so, I'm having a combat system and I tried doing touched event on the client on the user's limbs when needed and to damage with a remote event (ofc I'm running checks at the server before I let it damage) anyways, it worked perfectly but then something weird happened when 2 users were fighting eachother, when the player punched the other player and did a nice punch combo sometimes the player itself would get damaged which is really weird, so I moved the touch events from the client script to a server side script and now the problem is different, when you punch a player at the first few punches it won't damage the player like it should have, and it will only damage after a few tries, here is a part of the global and local script below, would be happy if one will manage to explain to me why this might happen:

Local script:

--// Services
local UserInputService = game:GetService('UserInputService')
local ReplicatedStorage = game:GetService('ReplicatedStorage')
local Players = game:GetService('Players')

--//Player
Player = game.Players.LocalPlayer
Character = game.Workspace:WaitForChild(Player.Name)
Humanoid = Character:WaitForChild('Humanoid')
RightArm = Character:WaitForChild('RightLowerArm'):WaitForChild("HITBOX")
RightLeg = Character:WaitForChild('RightLowerLeg'):WaitForChild("HITBOX")
LeftArm = Character:WaitForChild('LeftLowerArm'):WaitForChild("HITBOX")
LeftLeg = Character:WaitForChild('LeftLowerLeg'):WaitForChild("HITBOX")

--// Events
local Combat = ReplicatedStorage:WaitForChild('Combat')

--//PUNCHING + KICKING

UserInputService.InputBegan:Connect(function(Input,GameP)
	if GameP then return end
	if Input.UserInputType == Enum.UserInputType.MouseButton1 then
		local Word = "ATTACK"
		Combat:FireServer(Word,nil)
	end
end)

Script:

The touched event with a check before damage and a debounce