What folder does Aperture store everything in?

Everything gets stored in the Aperture library, not the application, with the possible exception of the original files.

They can be stored either in the library, this is called managed, or outside the library, called referenced.

Simple enough?

In other words, it seems the only way to get to anything I've downloaded, I have to open Aperture. This strikes me as kind of strange considering nothing else I've ever used works that way, but I certainly can't find any of the pictures and videos outside of opening Aperture.

Aperture is a management system for your image files. It stores the images in the Aperture library and retrieves them for you. If you use Aperture to access your images, you do not need to know where they are. If you access the images in any other way, you are likely to mess up Aperture's bookkeeping and it will not be able to find the images again.

Also, I then assume that if I delete anything in Aperture, it permanently deletes it from my computer instead of just deleting it from the program. Am I correct about all of this

Images that you import into the Aperture library and are managed by Aperture, will be deleted from the library, when you delete them in Aperture.

This User Tip by Terence Devlin describes in more detail the methods to access images in a Photo library like the Aperture library or iPhoto library:

How do I access my Photos in Aperture?

Also, it appears that at some point I moved some files out of Aperture and deleted them and now they're showing up in Aperture as not existing anymore, which really confuses me. If I copied those files elsewhere why would Aperture lose that file? I'm rather confused right now if you can't tell.

Don't move image files from the Aperture library using the Finder. If you want to relocate the images to a differnet location, let Aperture do it (File > Relocate ...). Only then will Aperture be able to update its internal databases and keep track of the image files.

To reconnect images that you moved from the library use Aperture's command "Locate Referenced Files". This will work only for referenced files. If you actually moved images out of the Aperture Library package, reimport the images and delete the old versions, that are showing as missing. See this Manual page: Aperture 3 User Manual: Working with Referenced Images