Custom mouse cursor getting closer to camera and not accurate?

I tested your script and it's a quite common problem as far as I've seen. Basically what's happening is the pointer keeps on coming closer and closer to the camera, and there's a rather simple reason why.

What's happening is that the first time you move the mouse, mouse.Hit returns a position on the ground that you were pointing at and the pointer moves to that location. The second time the RenderStepped event fires, mouse.Hit returns a position on the POINTER, instead of the ground. That's happening because the script doesn't know to ignore the pointer. There's a relatively simple solution to this whole mess though.

TargetFilter

The mouse object has a property called TargetFilter which is basically an object for the mouse to ignore. When you set an object as the mouse's TargetFilter, the mouse acts as if that object isn't even there, and just ignores it. You use it like so:

Mouse.TargetFilter = Part

Now that we know how to use the TargetFilter, we can put that in your script, like so:

--I moved all the static code to the top because those only need to be changed once. It's unnecessary to put them in a loop because they will always be the same. Not only that, putting it in a loop could cause unwanted lag
local pointer=Instance.new("Part")
pointer.Anchored=true
pointer.Parent=workspace
pointer.FormFactor="Custom"
pointer.BrickColor=BrickColor.new("Bright blue")
pointer.Size=Vector3.new(2,.2,2)

local player=game.Players.LocalPlayer
local mouse=player:GetMouse()

mouse.TargetFilter = pointer --This makes the mouse ignore the pointer

game:GetService("RunService").RenderStepped:connect(function()
    pointer.CFrame=CFrame.new(mouse.Hit.p) --Only this needs to be in the loop because this is the only thing that changes. All the other code remains the same
end)

Hope this helped!

More info: Mouse