I have this exact same issue. I don't use my iPod Touch 4th gen much anymore, but I figured it's a nice piece of technlology that could certainly have a purpose, all my podcasts are weekly podcasts that I listen to on the road during the week (1 hour commute to work) So I figured I'd load my iPod Touch on the weekends and keep it in the car the rest of the week.
However the need for this podcast application that I can't access is keeping me from actually having a use for this thing. I mean yea, I could just use some other podcasting program, I'm sure something like pocketcast probably exists for the iPod Touch, but I wanted to use the iTunes app.
To the person saying about how the app should give me an option to download a previous version, I understand that's what is supposed to happen, it certainly happened when I tried downloading something else in the past, but for Podcasts it doesn't happen.
Come on Apple! I understand not being able to get iOS 7, even though the 4th gen iPod Touch isn't THAT old, but don't make it useless, let me go back to a previous version of iOS so it runs smooth again and let me use podcasts again!