Help with swapping the position of two selected parts using remote events?

GIF of the current situation https://gyazo.com/1595731dce089ffa39d10e0786e7e50b

SERVERSCRIPT

game.ReplicatedStorage.Shambles.OnServerEvent:Connect(function(player, part1, part2)
		print("Teleported Parts")
		part1.Position = part2.Position
		part2.Position = part1.Position
end)

LOCALSCRIPT

play = game.Players.LocalPlayer
mouse = play:GetMouse()
local selectionbox = Instance.new("SelectionBox")
selectionbox.Color3 = Color3.new(5,0,0)
selectionbox.Name = "redbox"
selectionbox.Archivable = true
local selectionbox2 = Instance.new("SelectionBox")
selectionbox2.Color3 = Color3.new(0,5,0)
selectionbox2.Name = "greenbox"
selectionbox2.Archivable = true

function key(inputObject,gameProcessed)
	if inputObject.KeyCode == Enum.KeyCode.X then
		if mouse.Target == nil then end
		if mouse.Target ~= nil and mouse.Target.Name ~= "CutPlane1" and mouse.Target.Name ~= "CutPlane2" then
    		if mouse.Target.Name == "Baseplate" then end
			selectionbox.Parent = mouse.Target
    		selectionbox.Adornee = mouse.Target
    		wait()
		end
	end
end

game:GetService("UserInputService").InputBegan:Connect(key)


function key(inputObject,gameProcessed)
	if inputObject.KeyCode == Enum.KeyCode.C then
		if mouse.Target == nil then end
		if mouse.Target ~= nil and mouse.Target.Name ~= "CutPlane1" and mouse.Target.Name ~= "CutPlane2" then
    		if mouse.Target.Name == "Baseplate" then end
			selectionbox2.Parent = mouse.Target
    		selectionbox2.Adornee = mouse.Target
    		wait()
		end
	end
end
	
game:GetService("UserInputService").InputBegan:Connect(key)

function key(inputObject,gameProcessed)
	if inputObject.KeyCode == Enum.KeyCode.V and selectionbox.Parent ~= nil and selectionbox2.Parent ~= nil then
		print("Fired Teleport")
		game.ReplicatedStorage.Shambles:FireServer(selectionbox.Parent, selectionbox2.Parent)
	end
end

game:GetService("UserInputService").InputBegan:Connect(key)

From my understanding, the issue is that Part1 get's teleported to Part2, and then Part2 gets teleported to Part1 meaning they both end up in the same position, I'm trying to find a way to store their original positions and then teleport them with that, not sure how to go about it though. Help is appreciated.