How do I ignore hats in raycasting?

How can I make raycasts ignore all hats? If I used

if part:isA("accessory") then
--damage stuff here
end

Players with bigger hats would be at a disadvantage and it would be hard to detect headshots

What you are looking for is the FindPartOnRayWithIgnoreList function.

This function, as implied by its name, will return the BasePart that the specified Ray intersects, along with its point of intersection, surface normal at the point of intersection, and the material of the surface that was hit (if any such intersecting part is found, that is).

local ray = Ray.new(origin, direction)
local ignore = {} -- include parts to ignore in this table

local hit, pos, normal, material = workspace:FindPartOnRayWithIgnoreList(ray, ignore)

Please take a look at the Wiki page linked earlier to learn more about the optional parameters you can fill in while raycasting with an ignore list.