Help with swapping the position of two selected parts using remote events?

GIF of the current situation https://gyazo.com/1595731dce089ffa39d10e0786e7e50b

SERVERSCRIPT

game.ReplicatedStorage.Shambles.OnServerEvent:Connect(function(player, part1, part2)
        print("Teleported Parts")
        part1.Position = part2.Position
        part2.Position = part1.Position
end)

LOCALSCRIPT

play = game.Players.LocalPlayer
mouse = play:GetMouse()
local selectionbox = Instance.new("SelectionBox")
selectionbox.Color3 = Color3.new(5,0,0)
selectionbox.Name = "redbox"
selectionbox.Archivable = true
local selectionbox2 = Instance.new("SelectionBox")
selectionbox2.Color3 = Color3.new(0,5,0)
selectionbox2.Name = "greenbox"
selectionbox2.Archivable = true

function key(inputObject,gameProcessed)
    if inputObject.KeyCode == Enum.KeyCode.X then
        if mouse.Target == nil then end
        if mouse.Target ~= nil and mouse.Target.Name ~= "CutPlane1" and mouse.Target.Name ~= "CutPlane2" then
            if mouse.Target.Name == "Baseplate" then end
            selectionbox.Parent = mouse.Target
            selectionbox.Adornee = mouse.Target
            wait()
        end
    end
end

game:GetService("UserInputService").InputBegan:Connect(key)


function key(inputObject,gameProcessed)
    if inputObject.KeyCode == Enum.KeyCode.C then
        if mouse.Target == nil then end
        if mouse.Target ~= nil and mouse.Target.Name ~= "CutPlane1" and mouse.Target.Name ~= "CutPlane2" then
            if mouse.Target.Name == "Baseplate" then end
            selectionbox2.Parent = mouse.Target
            selectionbox2.Adornee = mouse.Target
            wait()
        end
    end
end

game:GetService("UserInputService").InputBegan:Connect(key)

function key(inputObject,gameProcessed)
    if inputObject.KeyCode == Enum.KeyCode.V and selectionbox.Parent ~= nil and selectionbox2.Parent ~= nil then
        print("Fired Teleport")
        game.ReplicatedStorage.Shambles:FireServer(selectionbox.Parent, selectionbox2.Parent)
    end
end

game:GetService("UserInputService").InputBegan:Connect(key)

From my understanding, the issue is that Part1 get's teleported to Part2, and then Part2 gets teleported to Part1 meaning they both end up in the same position, I'm trying to find a way to store their original positions and then teleport them with that, not sure how to go about it though. Help is appreciated.