Batteries eventually become worn out; even if you don't use the device that often, they deteriorate due to age. The best thing for a battery, if you don't use the device that often, is to "exercise" it. Make it a point to use the device, and drain the battery (then recharge it) at least once a month. Letting a device sit "in a drawer" for a long time (with a drained battery) can cause its lithium battery to no longer hold a charge. And always leaving it connected to power is not that great for the battery either. Replacement battery parts are available online, through sources like Amazon and eBay, even for older iPods. Some iPod models have a simple connector for the battery. Others require de-soldering and re-soldering of wires to replace the battery (usually for smaller devices like iPod nano and shuffle).
Hard drives in iPods also eventually become faulty, like hard drives in computers. They have delicate precision moving parts. Flash-based ("solid state") storage is better for use in iPods. Once the cost of flash-based storage came down, Apple switched to it for iPods instead of continuing with even higher capacity mini hard drives. Those hard drive components are no longer being produced; replacement parts are likely to be scavenged used parts. However, you can buy adapters that allow Compact Flash cards and SD cards (both common flash storage formats) to be used in iPods to replace faulty hard drives.
The 3rd gen "classic" iPod (from around 2003) has some limitations. While it has a 30-pin dock connector (like later iPods), it does not charge from a USB connection. It needs a FireWire connection to change and sync. Most PCs (past and present) do not have FireWire, and current Macs don't have FireWire either (although you can adapt a Thunderbolt port to FireWire). There is no such thing as a USB-to-FireWire adapter; if you see one online, it is a scam and may even cause damage to the connected device. There is a special odd-looking cable Apple used to sell (aimed at Windows PC users at the time) that allows connection of 3rd gen iPod to a USB port, to sync AND charge at the same time. It has a 30-pin dock connector, with BOTH a FireWire cable and USB cable coming out of it. The USB connector goes to the computer's USB port. At the same time, the FireWire connector goes to a FireWire power adapter (that came with these iPods). The iPod connects for data through USB while getting power from FireWire.
On the plus side, the 3rd gen iPod has a unique all-touch interface (this was before the click-wheel). The Hold switch is the only control that is not touch-based; important because it was too easy to mistakenly touch one of the playback control buttons, so the Hold switch was actually used. Today's "iPod touch" has more physical controls than 3rd gen iPod. 🙂 It also has excellent sound quality (to my ears).
A more practical old "retro" iPod for use today is the 4th gen iPod and iPod mini. They can connect and charge using USB, and they are the first iPods with the familiar and effective click-wheel interface (iPod mini actually had it first). They have monochrome screens that are easy to read, even in bright sunlight, and without using the backlight in normal lighting. 4th gen iPod sounds slightly better than iPod mini. The "micro hard drive" in iPod mini uses the standard Compact Flash card connector, and can be replaced by a CF card. No adapter needed (unless you want to use an SD card instead).