How can I rectify the camera jittering issue in this movement script?

Alright so, I made the code below that rotates the players camera depending on if they are moving to the left/right. However, whenever I move left with a, move forwards with w, and then move my camera to the left, the camera shifts back into its original position until I stop moving my camera. This isn't supposed to occur. (It also happens if you move backwards and right, basically the opposite as the example) Any idea why it might be doing that? (MUST BE IN FIRST PERSON MODE)

local camera = workspace:WaitForChild("Camera")
local movementDirection = camera.CFrame:vectorToObjectSpace(humanoid.MoveDirection).Unit
	if math.abs(movementDirection.X) > math.abs(movementDirection.Z) then
		print(movementDirection.X)
		if movementDirection.X < 0 then
			camera.CFrame *= CFrame.Angles(0,0,0.1)
		elseif movementDirection.X > 0 then
			camera.CFrame *= CFrame.Angles(0,0,-0.1)
		else
			print("not moving")
		end
	elseif math.abs(movementDirection.X) < math.abs(movementDirection.Z) then
		-- if z is negative then character is moving forward, else they are moving backward (in relation to camera)
		
		print("forward or backward")
	end
end)

It's been driving me up the wall lately and I just can't seem to figure out why it might be doing this.

ISSUE IN ACTION: https://gyazo.com/987c104f50c191d969ee3aecbb00fffa