This may be a common problem, but why does my scripted part move upwards when changing vector?

Use the built-in Code block function next time.

The only reason why it is not staying below is because the position property, rotation property, or any other properties that deals with Vector3 has a built-in collision check, which will place the brick that's manipulated by the position property on top of whatever it is colliding.

Say you have BrickA. Using the position property, you set it at a point within halfway of BrickB. The result is that BrickA will be placed on top of BrickB (or will be placed at a position where it doesn't go through any objects).

In your case, instead of using Part3.Rotation = Vector3.new(--[[Coordinates]]), consider using Coordinate Frame without fear of facing the collision-check:

Part3 = game.Workspace.Part3

local Debounce = false

function onTouch(part)
	if Debounce == false then
		Debounce = true
		Part3.CFrame = CFrame.new(Part3.CFrame.x, Part3.CFrame.y, Part3.CFrame.z) * CFrame.Angles(0, math.pi/2, 0) -- Notice how math.pi/2 is in radians. You may also do math.rad(45), with 45 given in degrees.
		wait(1)
		Part3.CFrame= CFrame.new(Part3.CFrame.x, Part3.CFrame.y, Part3.CFrame.z) * CFrame.Angles(0, 0, 0)
		wait(1)
		Debounce = false
	end
end
script.Parent.Touched:connect(onTouch)

For more information about CFrame, visit here.