How to detect player if player is 5 studs away?

There's a thing called magnitude. An example would be:

local distance = Vector3.new(0,10,0)
local distance2 = Vector3.new(0,-10,0)
local magnitude = (distance - distance2).magnitude

print(magnitude)

This would print "20" in the output.

For a more in-depth explanation view the Roblox Wiki.

Anyway, magnitude is used to find the distance between one point to the other.

In your case, you want to find the distance between a player and an enemy.

I won't write an entire script for AI but just use your current code and find a way to put this in:

local magnitude = (enemyName.Position - characterDetected.Position).magnitude
print(magnitude) -- you can delete this if you want

if magnitude <= 5 then
    characterDetected.Humanoid.Health = 0
end

I hope this helps!