Airpods are connecting to both iPad and iPhone at the same time?

I recently updated my iPad Pro and iPhone XS Max to OS 14.0 and noticed my Airpods Pro are showing as connected to both at the same time. The issue is if I'm listening to music on device and I get a notification on the other, the music shows it's playing, but nothing is actually playing. For example, I'm listening to Apple Music on my iPhone, the iPad will get a notification from a game and I hear the ding, then the music will stop on my iPhone but I can see the song progress bar is still moving.

It's annoying more than anything, but I thought connecting to more than one device was a manual process?