How to make part that will be removed when clicked on it while holding a tool?

Im just trying to make broom system that will work when you click on part when holding a tool the part will be removed?

You can use tool.Equipped, and that will give you the player's mouse, since click detectors do not work when you hold the tool, you should use the mouse.Button1Down and mouse.Target to find out which object the player is aiming at (when clicked). Then you can fire a remote event to the server in order to destroy the part (since the whole mouse part is in the local script, the mouse is nil in a server script).

So, to sum up, you can learn using these resources:

tool.Equipped

mouse.Button1Down

mouse.Target

Remote Events

Add a clickDetector to the part and add a script in it and write this in the script

script.Parent.MouseClick:Connect(function(clicker)
	local Character = clicker.Character
	
	if Character:FindFirstChild("toolNameHere") then
		script.Parent.Parent:Destroy()
	end
end)

Use this local script in the tool:

local Part = -- Reference the part in this variable

script.Parent.Equipped:Connect(function()
   local ClickDetector = Instance.new("ClickDetector)
	ClickDetector.Parent = Part
end)

script.Parent.Unequipped:Connect(function()
    Part.ClickDetector:Destroy()
end)

Put this normal script in the part you want to get destroyed:

local Part = -- Reference the part in this variable

if Part.ClickDetector then
    Part.ClickDetector.MouseClick:Connect(function()
    	Part:Destroy()
    end)
end

These scripts makes it so when it gets clicked (only if there is a click detector, which gets put into the part if the tool is equipped) it gets destroyed, and it won't work when it is unequipped, because it gets destroyed.