If you have not already, you can try doing a Reset on the iPod
A Reset restarts the iPod (like rebooting a computer). It's possible the iPod is just stuck on that screen for some reason. Since the battery may be low, you can do the Reset procedure while the iPod is connected to the computer.
You can also try doing a Restore in the iPod in iTunes. The Restore button is on the iPod's Summary screen in iTunes. This will erase the iPod, reinstall its software, and set it to default settings.
But to answer your question, you can replace the battery on "classic" iPods. If you want to do it yourself, it is easier on 4th gen and earlier. On 5th gen and later, the seam on the case is really tight and more difficult to open. This web site has DIY guides.
The part is available on eBay and other online sources.