Since this problem occurred, have you turned OFF the shuffle. If not, put the power switch on the shuffle in the OFF position for at least ten seconds. That is a Reset. Then, turn it back ON.
Make sure the headphones plug is completely inserted into the headphone jack on the shuffle, as shown in the pictures in this document.
http://support.apple.com/kb/TS3572
The fit can be tighter than expected, especially when brand new. The shuffle can use headphones with remote control buttons on the cord, and if the headphone plug is not completely inserted, maybe it is getting false control signals (for volume up).
If that does not help, you may want to do a Restore on the shuffle using iTunes. The Restore button is on the shuffle's Summary screen in iTunes. This will erase the shuffle, reinstall its software, and set it to default settings. That would fix corruption in the shuffle's software or settings.
And if that does not help, return it for a refund and get another one.