How do I connect my Airpods Pro to my iPhone ONLY!! I don't want it switching to other devices!!!

I had some music playing on my macmini before I left for town. When I got back I put my airpods in my ear to make a phone call and the macmini music started playing in my ear. Various unpredictable behaviour like this keeps happening and is EXTREMELY annoying. I only use the airpods for phone calls. Thats it.

I want to pair my airpods with my phone and not any of my numerous other iCloud devices. How to do it?

Thanks