HOW DO I STOP AUTOFILL?!

I have always manually managed the music on my iPhone because my entire iTunes library (about 4000 songs) would exceed the amount of storage on my phone (32 GB). Upon buying the phone I spent quite a long time sorting through my music and deciding which songs to add. However, today I had the utter misfortune of accidentally clicking on "autofill," and there doesn't seem to be a way to turn it off. I now have no control over which songs are on my iPhone, and most of the ones that are on there are just gray ghost files. I can't delete them while on my phone, and when I delete them while on the computer they seem to disappear but then are still on my phone. Also, autofill tried to add more songs than can fit on my phone, so the unwanted, autofilled ghost songs can't even download anyway. If I try to add a new song, it just becomes another ghost file. If I eject and plug my phone back in, it just tries to start loading the ghost songs again and then failing. So I don't have music, just a bunch of ghost files, I can't delete anyting, and I can't add anything new. I'm stuck and completely frustrated, as I have been with iTunes for similar issues countless times before.

I also haven't been able to find any solution to this ridiculous problem online. Several other people seem to have the same problem and no one has an answer.