how to create a frame with instance.new()?

you are creating the gui in startergui, because you are creating the gui on a key press, i assume this is DURING game, the startergui is basically just a storage service that clones all of its children into a player when he/she joins, move it to the PLAYERGUI instead, im also assuming you are using a localscript or else userinputservice wont work

game:GetService("UserInputService").InputBegan:connect(function(inputobject,gameProcessedEvent)
    --case
    if inputobject.KeyCode == Enum.KeyCode.Q then
        local file = Instance.new("ScreenGui")
        file.Parent = game.Players.LocalPlayer.PlayerGui
        local paper = Instance.new("Frame")
        paper.Size = UDim2.new(0,100,0,100)
        paper.Visible = true
        paper.Parent = file 
    end
end)

To make the frame visible, it's as simple as this:

local paper = Instance.new("Frame")
paper.Visible = true
file.BackgroundTransparency = 0 -- Just to make sure that it's really visible.

Oh, and I also suggest you make the ScreenGui's Enabled property to true by doing something like this:

local file = Instance.new("ScreenGui")
file.Enabled = true

Because the frame's visibility won't matter if the ScreenGui was Disabled. Also take note that if this doesn't work, it may be because you're using the StarterGui instead of the PlayerGui

Use PlayerGui instead of StarterGui. Example:

game:GetService("UserInputService").InputBegan:connect(function(inputobject,gameProcessedEvent)
--case
    if inputobject.KeyCode == Enum.KeyCode.Q then
        local file = Instance.new("ScreenGui")
        file.Parent = game.Players.LocalPlayer.PlayerGui
        local paper = Instance.new("Frame")
        paper.Size = UDim2.new(0,100,0,100)
        paper.Visible = true
        paper.Parent = file
    end
end)