I think turning itunes match off while it is still trying to load something causes hiccups. So you need to start fresh again as best you can.
Here is what I did. First I went to Settings>General>Usage, then in the storage section, I tap on Music app, then hit the edit button and then delete(This will delete locally store music so be sure you have a backup in the cloud or on your computer first). Now I turned ON itunes match on the iPhone while connected to wifi. Then I opened the Music app and waited for the pinwheel to go away(loading album art etc). So at this point, there is no physical music on the iphone, however I have access to and see all of my music in itunes match available for download. Then I went back to settings and turned OFF itunes match on the iPhone. If this has worked correctly and everything is gone, it should say "No Content, You can download music from iTunes" when you open the music app. If it does not or you see a playlist or genius or anything, then there is still something left over.
From here, I chose to sync my music via itunes and leave itunes match off on the iphone. You may want to try turning itunes match back on the iphone if thats what you are wanting.
