Anybody have issues with resuming music/podcast playback while using airpods/other headphones?

I have had this issue since using any type of wireless headphone. I have a few different sets and they all do this, even the airpods. The issue is I'll pause the music or podcast that I'm listening to and then when I go to resume playback by using the force sensor or tapping play on the other headphones (Samsung/Skullcandy) the music will start for a split second and then pause again and then I'll have to hit the sensor again to start playback. It's really annoying and I'm sure it isn't supposed to work like this considering if I were to just hit the play button on the phone itself it will work just fine. It seems to only happen with wireless headphones. Anybody have this issue or a fix for it? I've done resets on the airpods and it doesn't do anything. I've tried with the screen off and with the screen on and it still does the same thing. Plays, pauses immediately then I have to hit play again. Any help would be greatly appreciated!