From the user's guide:
Delete a song from iPod touch: In Songs, swipe the song, then tap Delete.
The song is deleted from iPod touch, but not from your iTunes library on your Mac or PC, or from iCloud.
When iTunes Match is turned on, you can’t delete music. If space is needed, iTunes Match removes music for you, starting with the oldest and least played songs.
You can delete songs that you downloaded to your device if you find that your device is running low on local storage space.
- Open the Music app on your device.
- Tap Songs from the bottom navigation bar.
- Scroll or search for the songs you would like to delete from your device.
- Swipe left or right on the songs, and then tap Delete.
Note: You must be in Songs view to delete songs from your device.
I do not have iTunes Match turned on, all my songs are downloaded onto my iPod. When I am in the music app, and I swipe a song on the list, it does not pop up with a delete button. Instead of displays the full song information with a little arrow where my finger is. This does it when I sort by song, album, or artist. I am running iOS 6.1
Please actually try to do this before responding.