I FINALLY FIGURED OUT HOW TO DO IT!!!
I had the same problem when I bought a new iPhone.
The bigger and more complicated problem is that since it hasn't been activated yet, you can't get to your voicemail the old-fashioned way.
Here's how to do it:
1. Try the simple stuff; restart and reset. (DO NOT RESTORE YET, IT TAKES TOO LONG, AND IT MIGHT BE SOLVED EASILY WITH SOME OTHER STUFF)
2. Call AT&T (800-331-0500). Enter your phone number, press 3, and press 3 again (don't press 1 and 2, they won't help with your problem). Now you have "tricked" them into setting up your voicemail.
3. You will receive a text with your new voicemail password. Access your voicemail the old-fashioned way (preferably from a different phone) to complete the setup. Now at least you have some access to your voicemail.
After setup of your voicemail, try tapping the voicemail tab, it should work. If it doesn't, try restoring your phone. If you are still having problems, I can't help you, but at least you have some way of accessing your voicemail.