How do I delete audio files from Apple Music and iPhone?

I download audio files from interviews onto an external hard drive. My computer uses Apple Music to play these files, and then automatically puts them in my music library, which then automatically syncs them to my iPhone. So then every time I sync music, I have hundreds of audio files that I do not need on my phone. The only way I've seen to remove them is by selecting them one at a time on my phone and deleting them. deleting the group of them from Apple Music is not working.

Am I missing some process here? This takes forever, and I have to do it every time I sync my phone. I'd rather sync all my music because if I select by other methods like artist, some of my music that I DO want to listen to gets left off. These files don't show up with artist or song info.