I was having this SAME problem with and iPod Touch 4th gen I just purchased. 😠 It was driving me crazy. I went through all of the suggestions I could find on here and although KithKanen mentioned "go to the App Store and view your account details (menu: Store | View This Account)" I still couldn't find how to do that (call me lame) 😉 but FINALLY this is exactly what I did that worked, which boils down to the same thing but here are the actual steps after you've synched your device and done all the obvious on the computer:
1. Go to 'Settings' on your device
2. Go to 'Store'
3. Touch your "Apple ID"
4. The options "View Apple ID" or "Sign Out" come up
5. Choose "View Apple ID"
6. Edit payment information
7. Your credit card verification number is most likely blank because it's a new device. Enter it. Click 'Done' (I think, can't see it now because it won't let me in)
8. Should be all good.
Now I will say that I've tried going back to it and that stupid box keeps popping up as if I didn't enter my password BUT I am now able to download and make purchases. I tried updating my address to make it match the USPS and Google maps, I entered my credit card verification online in the iTunes store while my ipod was synched to the CPU, I changed my password (which now requires one capital, one number, and lowercase. I never used the capital before and never had a problem on my iPhone so who knows but I changed it because I was at the end of my rope), I made sure it had a title chosen (i.e. Mrs.), and NONE OF THOSE worked. Or maybe they all worked in conjunction, who knows, but the final straw was what I listed above. Phew!