deleting songs

I have an iPhone 5 with the latest software update. There are songs in my music library that show up in the music player on my phone, but do not show up as "on this phone" when connected to iTunes. Also, I cannot delete these songs from my iPhone. I have checked my setting and "show all music" is not turned on. How can I delete these songs? If I try to swipe them to delete them they only start to play.