You should visit the Genius Bar before you return it, if you have an Apple Store nearby. You can make a reservation on the store's web page.
http://www.apple.com/retail/
Then, you can return it right there (assuming you bought from Apple directly), if you desire. OR, they might fix the problem. OR, if there is a defect in the shuffle that you got, you can get a replacement.
Within the first 90 days after purchase, you can also contact Apple's tech support by phone. Go to the Contact Us link at the bottom right corner of this web page.
If you want to try something right now, connect the shuffle to your computer and run iTunes. Select the shuffle in the iTunes sidebar, under DEVICES. Over to the right, go to the Summary tab. There is a Restore button there. It will erase the shuffle, reinstall its software, and set it to default settings. If there was a software or settings glitch causing this problem, that may fix it.
FYI - To make VoiceOver go through the playlists, you need to press and hold the VoiceOver button. This document has detailed information about using VoiceOver on the 4th gen shuffle
http://support.apple.com/kb/HT4322