I want to make a part that will open a gui when touched, and close it when you leave the bounds of the part. So far I've tried making a table for each player and adding/removing parts from this when they leave the part, but this is glitchy at the edges of the part. I've also tried checking for HumanoidRootParts in a Region3 area, but this does not account for rotation.
So I see you have tried using Region3 but the problem was that it didn't take rotation into consideration. There is a module made by EgoMoose which can be used to make rotated Region3s. It can be found on this Devforum thread; hope it helps you out.
On the topic of detecting when a touch ends, I think using a Region3 is the best approach in general, seeing as Roblox's own .TouchEnded
event is notorious for being unreliable - which is a shame because it would be the standard solution to the problem if it actually worked - you could try your luck with it, but again I think the module is your best bet.