How do I access the PlayerGui of a player that is NOT the LocalPlayer?

Okay to break this down a bit more:

I'm trying to create a group invite system where the LocalPlayer sends an invite to a regular player in the server and a GUI pops up on the invited player's screen. They can either click accept or decline and right now, I'm just trying to create a MouseButton1Down function for when they click accept. I can't use a Local Script to code this because PlayerGui does not exist for any player except the LocalPlayer. I also can't seem to use a server script because you can't manipulate GUI's with just server scripts. I've tried to create a remote event where I define PlayerGui as a variable in a server script and then transfer that variable over to a Local Script where I can use that variable to manipulate the GUI, but none of that worked for me.

I feel like there is an easy solution to this but I can't quite grasp it.

You can access another player's Gui with this variable local playerGui = players.PlayerName:WaitForChild("PlayerGui"), though you can store another variable or make a function to automatically change the PlayerName inside the playerGui variable. But there might be another ways of accessing another player's Gui. So, who knows?