I was having this problem with a bunch of music my daughter purchased on my wife's account. Somehow, I suppose because our Macs are all linked together, and maybe because I let her download a couple of tunes on my account for her iPods, over the years. this stuff was showing up in the cloud under the artist and songs list on the Music app on my new iPod Touch. What finally worked for me was, I think, a couple of things.
1st) I went to Settings > iTunes & App Store, and turned off "Show All" on Music and Videos.
2nd) I went to General > Usage > Manage Storage > Music, and deleted everything. When I went back to the Music app, everything was gone except my Playlists, that then showed as being empty, plus these songs and artists that were, evidently, on some cloud out there, somewhere.
3rd) I went to the iTunes store on my laptop, went into the list of "Purchased" items under my account, and there saw all the stuff I wanted to get rid of. Well… when you hover your mouse or select a song or album, there's an "X" in a little circle that pops up. Clicking that "X" will hide that item in the "Purchased" list. And, likewise, that item disappeared from the songs and artist list on the iTouch. Hallelujah!
4th) I went back and synced the music from my laptop back to the iTouch that I actually wanted to have.
Done!