I have just spent an hour solving this. IF you are someone who is having trouble setting up yours, maybe this will help. When I first went to Phone > Voicemail, it auto-dialed the voicemail number and said that this person has not set up their voice mail box.
Here is how I solved the problem:
Logged into my account on the ATT site, went to My Services, chose my phone (in case you have more than one on your account) and chose "Reset My Password." http://www.wireless.att.com/
About 1-2 minutes later, it sent a text message confirming the reset to my phone.
I waited about 5 minutes then went again on my phone to Phone > Voicemail. Finally now there was a blue button asking me to set up my voice mail. At that point it was very easy to set up a new password, and the next screen took me to the place to record a custom greeting.