Some USB ports (at the front of the PC, USB Hubs, one of the Notebook ports) are low powered and don't have enough juice for your iPod. The Wall charger maybe faulty, unless it can charge another iPod successfully.
Normally it will take at least two hour, to charge the iPod to 80% capacity.
Read this Apple Support link for your case.
But if your iPod is still under warranty, (used less than 20 times?) bring it in to Apple Store.
Then perhap your iPod battery is dying, Apple recommends that iPod battery should be fully charged at least once a month, or else the recharge cycles is shorten.
But have you Reset the iPod, by Pressing Menu and Center Button for about 10 secs till the Apple logo Comes ON then release the buttons.
Here is another support doc on Battery problems, I know, you would most probably have gone through it.