How inefficient is my use of FilteringEnabled?

I have created a gateway between the server and the client for the Dragger class. This has allowed the player to use building tools once again but the I am very aware I am overdoing my use of RemoteEvents. In a period of several seconds a player may send up to hundreds of requests to an event which handles the movement based on the UnitRay cast of their mouse.

Suppose I have 30-40 players sending thousands and thousands of requests a minute. This is an obvious glaring issue with the system. I just cant seem to find a workaround.

Suggestions?

Relevant Bits of Code:

ServerEvent:

DraggerMove.OnServerEvent:connect( function( Player, Data )
	local Dragger = Draggers[ Player ]
	
	if Dragger then
		local MouseRay = Data["MouseRay"]
		
		Dragger:MouseMove( MouseRay )
	end
end)

Call to Event:

	Mouse.Button1Down:connect( function()
		local Target = Mouse.Target
		local Hit = Mouse.Hit
		
		if Target ~= nil and Target.Locked == false then
			local TargetCFrame = Target.CFrame
			local PointOnPart = TargetCFrame:toObjectSpace( CFrame.new( Hit.p ) ).p

			***[excluded]***

			-- Activate Dragger --
			local DragData = {
				["Target"] = Target,
				["PointOnPart"] = PointOnPart,
			}
			
			Storage.DraggerDown:FireServer( DragData )
		end
	end)

One solution would be only sending events for when a player clicks/grabs a brick, and when the player lets go of it. Thus, you wouldn't need to fire events for every single movement, but the brick would still be moved.

When a player clicks/holds on to a brick, you could fire an event to the server so that no other player can grab hold of it (the server could turn on the Locked property.)

Then, have every single dragging movement done on the client.

When the player releases their mouse, you could fire an event telling the server to move the brick to the new position and to MakeJoints().

It'd look like the brick jumps around for other players, but at least it's not a ton of events firing.