I'm sorry to see that so many of you here are still having this problem. As far as I know there is still no permanent fix, hopefully iOS 5.1 will fix it when it is released. However, in my case the temporary fixes I mentioned earlier in this thread have consistently worked so I would others to try them again: I first noticed this low volume problem with my iPhone 4 after upg to iOS 5.0.0/5.0.1. Both the fixes i mentioned earlier worked on the iPhone 4. I then bought a iPhone 4S and after a few weeks of use, this too started exhibiting this low volume bug. On both the phones, both the temp fixes worked every time so please do try them again, you might get lucky:
1) Fix #1: while playing music through headphones, go to the Voice Memos App. Launch the App and you will notice that the music fades out and will be paused. Now you do not need to actually record any memo, just let the Voice Memo App run for 3 or 4 seconds and then exit the App. Now after exiting the Voice Memo App, go to the Music app and play the music. It should now play at proper volume.
2) Fix #2: While playing music throught headphones, go to Settings App > General > International and change the Language to Any other language. For instance, if you are in English, then change to British English or vice versa. Once you do that, you will notice that the Springboard relaunches. After the Springboard relaunches, go to the Settings App again and change your language back original language. Once again, your Springboard will relaunch. Now try the Music, it should be at proper volume now.
Of course as a few people have indicated here, there could be a hardware issue causing this bug too, such as dirty headphones contacts etc. But if your device is fairly new and you are having this problem, then it is most certainly a software bug and I do hope the suggestions above will help at least some of you... Best wishes.