Audible starts playing every time my airpods connect

I have an iPhone XS Max and AirPods Pro. Everytime I put my AirPods on Audible starts automatically playing. I keep losing my place in books because I don't always notice. Or I will get a call, I will grab my airpods and when I am done with the call I will put them back in the case, what will happen is audible starts to play and I have no idea because the airpods are in the case. This is incredibly frustrating as I do not want anything to automatically start just because I took the airpods out of the case. The app is closed by the way so it is not because it was open. How do I stop the automatic playing of ANYTHING just because I took the airpods out?