CFraming a model without colliding into other models?

So I'm trying to make a model CFrame and when it touches the brick that has this script, the value I made up and is named Anchor turns to true and the model goes to Vector3.new(-20,0,0), everything is working fine, the problem is that this script breaks if a player and the model are touching this brick at the same time.

Could anyone think of a solution to fix it from breaking, or maybe another way I could do what I'm trying to do yet more efficiently?

while wait(1) do
local Stop = script.Parent:GetTouchingParts()
for i = 1,#Stop do
	
local model = Stop[i].Parent
local part = model:WaitForChild("PrimaryPart")
model.PrimaryPart = part
model:SetPrimaryPartCFrame(part.CFrame+Vector3.new(-20,0,0))

if Stop[i].Parent:FindFirstChild("Settings") then
Stop[i].Parent.Settings.Anchor.Value = true	

elseif not Stop[i].Parent:FindFirstChild("Settings") then
end	
	
end
end