So after the latest iOS 9 update (9.0.2), despite Apple's saying they resolved some Podcast app issues, the application was still kicking me out. Troubleshooting seemed to indicate to me that doing a hard restart (double-tapping the home button and exiting out of the apps, then holding the power button and home button until I see the Apple symbol again) helped occasionally, but that when it came to playing podcasts, turning off WiFi was the most helpful. But I wanted to download new episodes of some of my podcasts, and I couldn't very well do that without WiFi. So I kept trying. It seemed like I could enter the app without getting kicked out when the WiFi was off, so I tried turning of WiFi, entering the app, canceling all current downloads, then swiping up from the bottom of the screen and turning the WiFi back on. It didn't kick me out, and I was able to download new episodes without any trouble. I had been trying the hard restart over and over again, and it didn't seem to be doing anything, but it may have. So that's what I would recommend:
1. Hard restart
2. Turn off the WiFi option
3. Start the app.
4. Try listening to something. If it doesn't kick you out, great.
If you want to download new episodes, try turning off the WiFi, getting into the app, canceling any downloads available, then turning WiFi back on and trying to download something.