I now have an "All Mail (Google)" folder in my Mail application with over 3,000 Emails. I'm wondering if those are now all being uploaded to my system, or if this is somehow a link to to the Gmail server. It's like Gmail never removed any of my old Emails at all.
Log into your Gmail account in Safari or your preferred browser.
Under Settings, click on Labels.
Uncheck “All Mail"
[Note: if you want to clean up All Mail messages it's easier to do in Mail than in your browser. Select a message and add it to VIPs. This will show all the messages from that sender. Select all and delete. Empty the trash and do the next message,]
Now to why you have all that old mail in the All Mail folder....
In Gmail, Trash and Delete are not the same thing to Google.
Deleting a message will not delete the message. It simply moves it from Inbox to All Mail. To really delete, you have to move the message to the Trash.
Mail should now be set to move your messages to Trash. Test by deleting a message and verify in Safari that it actually ended up in Trash. If not, you will need to tweak your Server settings in Mail.
You might want to simply delete all the older messages in All Mail to clean up. Otherwise, see the Note above for cleaning up in Mail.