Why are welds changing the position of the parts?

The weld object has two properties, called C0 and C1 which is the offset of part0 from part1 and the offset of part1 from part0, respectively. By default, both properties are CFrame.new() so one of the parts is moved to reflect this.

The solution is pretty simple:

1. Find the offset of part1 from part0. You can do this on the command line by: print("CFrame.new("..PartToWeldTo.CFrame:ToObjectSpace(Part.CFrame)..")")
2. Create the weld, and set C1 to the offset you found above.

It's that simple and it's also time consuming. You can use WeldConstraints for this like BulletproofVast said, but if you don't want to then I have a plugin for this.

Put this in a separate script, right-click the script and Save as local plugin..

local toolbar = plugin:CreateToolbar("Toolbar")
local weldPart = toolbar:CreateButton("Weld Part(s)", "Weld all parts to the first selected part.", "rbxassetid://4459262762")

weldPart.Click:Connect(function()
	local selection = game:GetService("Selection"):Get()
	local partToWeldTo = selection[1]
	
	for _,part in pairs(selection) do
		if part ~= partToWeldTo then
			local weld = Instance.new("Weld")
		
			weld.Part0, weld.Part1 = part, partToWeldTo
			weld.C1, weld.Parent = partToWeldTo.CFrame:ToObjectSpace(part.CFrame), weld.Part0
		end
	end
end)