I too have struggled with this. I tried this solution to no avail. I was able when deleting to remove a number of albums but when deleting a song from several albums a song started playing & now those items cannot be deleted by swiping right to left. They just keep playing.
I had to do a complete delete of all music in February & redownload only those I wanted currently on my phone. Very cumbersome, not an ideal solution. It should not be this difficult.
I was hoping with the recent IOS7 update that this would be resolved.
I have a large iTunes Library with iTune Match - 17,000+ songs. I only use my iPhone 4 for recently acquired material to listen to on the go, so need to constantly add/delete from phone.
I repeat this should not be this difficult.
It's a miracle. I powered off device items were still there. Wrote above & went back & the items I was trying to delete were gone.
I'm leaving iTunes Match disabled on my iPhone & manually manage as before dropping in to the device plugged in through iTunes. It seems to be working.
A little bit of this & a little of that.
Did you enable iTunes Match on your iPhone? If so, disable it by going to Settings > iTunes and App Store > iTunes Match > OFF
In addition, you can delete songs from any iDevice using copytrans manager without the need to sync/unsync with iTunes.