What I think Apple assumes we'd want to do: subscribe on all your devices, and set your devices to sync subscriptions. But I don't recommend that - I've found that the syncing doesn't work very well, and episodes appear and disappear in annoying and unpredictable ways.
What I do: I only subscribe on the device I want to use to listen to that podcast. On my iPhone I subscribe to all the podcasts that I listen to and dispose of. On my desktop I subscribe to the "permanent" podcasts that I want to keep - I change those to "Media Kind = Audiobooks", and they wind up synced to my iPhone into the iBooks app. I don't use podcasts on my iPad at all.
Where are you listening to podcasts? On one device only, or on several devices? If on one device only, I'd recommend not subscribing on other devices. If across several devices, you may want to try subscribing on all, and syncing subscriptions, but that can make the library do rather unpredictable things.