I just recently had issues with my phone where after I had deleted all my music from the device, it was still showing purchased content, including the **** U2 album. I had all the right things turned off, and everything was still there. And of course, I couldn't swipe and delete. So I decided to try things my way.
I opened iTunes and logged into my account.
iTunes >> iTunes Store >> Purchased
It was in my purchase account that I realized when you hovered over the content, a little X appears. This X allows you to HIDE, not delete, but HIDE the music from your phone.
After doing this to all the songs still on my phone, I watched them disappear from my phone as I "hid" each item. Including the "U2" album nobody knows how to get rid of. So I hope this helps.