Apple music app starts playing when unpausing the podcast app

This is a copy of a question that nobody answered. I reposted it because it happens to me a lot and it's annoying.

"It’s not just the podcast app, this happens if I’m listening to audible or Spotify, or Pandora, whatever. Sometimes (when listening with wireless beats) I have to pause what I’m listening to, to hear someone talking to me or pay attention to something else. When I try to go back to listening again by pushing the button on the headphones it ALWAYS starts playing a song from the music app that I was never listening to previously. It’s gotten to where I feel irritated by people trying to communicate because it means I’m going to have to dig my phone out of my pocket and hunt down the podcast again in order to continue listening.

Is there a setting that will stop iOS from defaulting back to music app when I unpause Spotify?"