I'm trying to figure out what exactly I'm looking for and where I find it. I tried clicking develop and configure game, but I don't see anything that has to do with data stores.
You would have to go to the Games tab, select whatever game you are working with, and check the Enable Studio Access to API Service, which would then allow you to access the data stores in studio sessions.