Help - I can't delete Visual Voicemails. They reappear after deleting.

I have an iPhone 5, iOS 7.0.4, AT&T.

When my voicemails come in, I listen and delete them. Then I go to "deleted items" and find that the phone has a hard time enabling "Clear All" - it is often greyed out, and I have to delete them all one by one. Sometimes I get a blue "Clear All" and am able to empty the box of all the voicemails at once.

Either way I empty that box - all at once through "clear all" or one by one - they often return to the "deleted items" box. Sometimes they even return to my normal voicemail box and show "unread" or "new" like I haven't listened to them at all.

Anyone else having this trouble? My software is up to date. Now what?