Why does my brick go inside objects facing a certain way in my placement system?

The side represented in the "blue outline" is where the blue block functions normally. The "red outline" is where the blue block goes inside the 10x30x10 brick as seen in these screenshots:

Blue - http://prntscr.com/kjxg4i

Red - http://prntscr.com/kjxg9f

I snipped out keybinds, if statements, etc. just to "clean" it up or make it simpler to read. It's just mathy stuff so the irrelevant code shouldn't matter.

Code -

-- LocalScript
local UserInputService = game:GetService("UserInputService");
local RunService = game:GetService("RunService");
local mouse = game.Players.LocalPlayer:GetMouse();

local offset = 5;
local part;

local function createObject()
	local newObject = Instance.new("Part");
	newObject.Size = Vector3.new(5, 5, 5);
	newObject.Anchored = true
	newObject.TopSurface = Enum.SurfaceType.Smooth
	newObject.BottomSurface = Enum.SurfaceType.Smooth
	newObject.CanCollide = false
	newObject.Name = "Hologram"
	newObject.BrickColor = BrickColor.Blue()
	newObject.Transparency = 0.3
	newObject.Parent = workspace
	return newObject;
end

local function calculateXYZ()
	local X_FLOOR = math.floor(mouse.Hit.p.X);
	local Y_FLOOR = math.floor(mouse.Hit.p.Y);
	local Z_FLOOR = math.floor(mouse.Hit.p.Z);
	
	local X = (X_FLOOR) - (X_FLOOR % offset) + (part.Size.X / 2);
	local Y = (Y_FLOOR) - (Y_FLOOR % offset) + (part.Size.Y / 2);
	local Z = (Z_FLOOR) - (Z_FLOOR % offset) + (part.Size.Z / 2);
	
	return X, Y, Z;
end

part = createObject();
mouse.TargetFilter = part
RunService.Stepped:Connect(function()
	print(mouse.TargetSurface)
	part.Position = Vector3.new(calculateXYZ());
end)

I'm pretty sure it has something to do with the "calculateXYZ" function but I have no clue at this point. My brain doesn't work q.q