Settings>General>Usage>Manage Storage>Music> is where you can delete albums and songs. Swiping across songs in the songs list still works for me. Perhaps you should try closing the Music app and reset the iPad.
In order to close apps, you have to drag the app up from the multitasking display. Double tap the home button and you will see apps lined up going left to right across the screen. Swipe to get to the app that you want to close and then swipe "up" on the app preview thumbnail to close it.
Reset the iPad by holding down on the sleep and home buttons at the same time for about 10-15 seconds until the Apple Logo appears - ignore the red slider if it appears on the screen - let go of the buttons. Let the iPad start up.