Floating book not orienting properly, anyone know why?

Maybe the book's LookVector (or -ZVector) is one of the sides. To make it face the right way, you need to make -XVector as new ZVector and ZVector as new XVector using CFrame.fromMatrix().

game.Players.PlayerAdded:Connect(function(Player)  
	Player.CharacterAdded:Connect(function(Character)
		if Player.TeamColor == BrickColor.new("Electric blue") then
			Character.Humanoid.MaxHealth = 350
			Character.Humanoid.Health = 350
			local mageBook = game.ReplicatedStorage:WaitForChild("FX").MageBook:Clone()
			mageBook.Parent = Character
			local bodyPos = Instance.new("BodyPosition", mageBook)
			bodyPos.MaxForce = Vector3.new(400000, 400000, 400000)
			local bodyGyro = Instance.new("BodyGyro", mageBook)
			bodyGyro.MaxTorque = Vector3.new(400000, 400000, 400000)
			while true do
				local headCF = Character.Head:GetPivot()
				
				bodyPos.Position = Character.HumanoidRootPart.Position + Vector3.new(1,1,4)
				bodyGyro.CFrame = CFrame.fromMatrix(headCF.Position, headCF.ZVector, headCF.YVector, -headCF.XVector)
				task.wait()
			end
		end
	end)
end)

Alternatively, you can rotate it by 90° on y-axis using CFrame.Angles().

game.Players.PlayerAdded:Connect(function(Player)  
	Player.CharacterAdded:Connect(function(Character)
		if Player.TeamColor == BrickColor.new("Electric blue") then
			Character.Humanoid.MaxHealth = 350
			Character.Humanoid.Health = 350
			local mageBook = game.ReplicatedStorage:WaitForChild("FX").MageBook:Clone()
			mageBook.Parent = Character
			local bodyPos = Instance.new("BodyPosition", mageBook)
			bodyPos.MaxForce = Vector3.new(400000, 400000, 400000)
			local bodyGyro = Instance.new("BodyGyro", mageBook)
			bodyGyro.MaxTorque = Vector3.new(400000, 400000, 400000)
			while true do
				bodyPos.Position = Character.HumanoidRootPart.Position + Vector3.new(1,1,4)
				bodyGyro.CFrame = Character.Head:GetPivot():ToWorldSpace(CFrame.Angles(0, math.rad(90), 0)) -- or Character.Head:GetPivot() * CFrame.Angles(0, math.rad(90), 0)
				task.wait()
			end
		end
	end)
end)