Fly script inverts my controls if I face backwards?

I'm trying to make a fly script that uses CFrame.LookVector and CFrame.RightVector and if works fine if I only face forward, however if I turn around my controls are inverted (W makes you go backwards, S makes you go forward etc). I have a video that is not very clear but hopefully you will see what I mean. This is my code:

local plr = game.Players.LocalPlayer
repeat wait() until plr.Character
local char = plr.Character
local hrp = char:WaitForChild("HumanoidRootPart")
local hum = char:WaitForChild("Humanoid")

hum.PlatformStand = true
hrp.Anchored = true

game:GetService("RunService").Stepped:Connect(function()
    hrp.CFrame = CFrame.new(hrp.CFrame.Position, hrp.CFrame.Position + workspace.CurrentCamera.CFrame.LookVector)
    if game:GetService("UserInputService"):IsKeyDown(Enum.KeyCode.W) then
        hrp.CFrame = hrp.CFrame * CFrame.new(workspace.CurrentCamera.CFrame.LookVector)
    end
    if game:GetService("UserInputService"):IsKeyDown(Enum.KeyCode.A) then
        hrp.CFrame = hrp.CFrame * CFrame.new(workspace.CurrentCamera.CFrame.RightVector):Inverse()
    end
    if game:GetService("UserInputService"):IsKeyDown(Enum.KeyCode.S) then
        hrp.CFrame = hrp.CFrame * CFrame.new(workspace.CurrentCamera.CFrame.LookVector):Inverse()
    end
    if game:GetService("UserInputService"):IsKeyDown(Enum.KeyCode.D) then
        hrp.CFrame = hrp.CFrame * CFrame.new(workspace.CurrentCamera.CFrame.RightVector)
    end
end)

I wasn't getting any errors in the console so I don't know how to fix it.