The most frustrating thing for me is that you can no longer hold down a song/album to see the full title if the title is longer than the screen. If you listen to a lot of classical music, you know how important this is, since in classical music, titles are often very long and differ only in the last few words:
Piano Sonata No. 1 in F Minor, Op. 2, No. 1: I. Allegro
Piano Sonata No. 1 in F Minor, Op. 2, No. 1: II. Adagio
Piano Sonata No. 1 in F Minor, Op. 2, No. 1: III. Menuetto (Allegretto)
Piano Sonata No. 1 in F Minor, Op. 2, No. 1: IV. Prestissimo
iPhone's screen can only fit "Piano Sonata No. 1 in F M" and there is no anymore an easy and convenient way to see the full title. This also applies to Playlists view, as most of the space is now taken by album art.
Also, you can't delete entire albums anymore if you want to free space. You have to do it song by song. Guess what, I have complete works albums (bought from iTunes) with hundreds of songs. It will take while deleting them track by track. The lost functionality for no good reason is very frustrating.
By the way, the design flaws of the new Music app were actually known for a long time. On MacRumors forums, there is a 14-pages topic about that started back in June: http://forums.macrumors.com/showthread.php?t=1601540&highlight=music+app