Can I configure my Airpods to ONLY connect to my phone and NEVER my iPad?

I have a phone and an iPad. I brought my iPad with the intention that I could play music for my napping son on my iPad while listening to audiobooks (with Airpods) on my phone. But my iPad was automatically connecting to the Airpods when I put them in. When I told my iPad to "forget this device," it automatically disconnected my phone.

Which means it's controlled by account and not by device. I only only ONLY want my Airpods to connect to my phone, full stop.

Worse still, while flying last week, every time I opened my Airpods, my son's iPad (on his own account, connected by Family) prompts him, "Do you want to connect to this device?" This pauses his show and he doesn't know how to deal (he's only 3).

How can I prevent my Airpods from trying to connect to every Apple product in range? These should never try to connect to any random device when I'm not holding down the Pair button. This should be an extention of my iPhone and that's it.