This may be a bug for all that I know but I am too cynical to believe that.
From everything that I have read, it appears as though Apple did remove the feature. What the reasons could be for that are beyond me. And I did find the paragraph that states that feature exists in the iPad. I copied this from the iPad User Guide for iOS 5. The free book that I downloaded in iBooks.
"Display a song’s lyrics
Display the album cover by tapping the thumbnail at the top of the screen. Lyrics appear if you’ve added them to the song using the song’s Info window in iTunes."
However, who knows with Apple anymore. Case in point - Apple changed the wording on their website about multitouch gestures being one of the new features of iOS 5, a day or two after the update came out. They decided to release the feature to work on the iPad 2 only - even though the website promoted the feature as being one of the new features for the "iPad" for months leading up to the release - leading all users to believe that it would work on the iPad 1 as well. I don't know if you read any of those threads but the screaming is finally starting to die down on that one.
My point is that even thought the guide says that the song lyrics works, I have not read of anyone that can confirm that it does. Maybe someone else will confirm that the feature is in the iOS, but I don't believe that it is. So for the time being - for me anyway - it is not a bug - but another "why did they do that" decision by Apple.