Stopping one iPad from playing music on HomePod

Apple household here. 2 iPhones, 2 iMacs, 1 MacBook, 3 iPads, 3 HomePods between my wife, 10yo son and I. My son has one of the iPads for his use, using my Apple ID but with ScreenTime to keep him from watching videos of Roblox and Minecraft on YouTube all day.

While I have a long list of gripes on ScreenTime to support the above kid scenario, one particular case is driving me crazy: when he plays one of his annoying songs on his iPad, it will starting playing on one of the HomePods, often replacing the music I'm listening to.

I've turned off AirPlay/HandOff on his iPad but it still happens - he's not TRYING to play on the HomePod, it just happens. Can't figure out how to disconnect his iPad from controlling the HomePod (because, I assume, his iPad is ultimately "mine" and I want my other devices to still be able to control the HomePod.)

BTW, similar problem when I have my AirPods in, it will occasionally transfer to me which is next level annoying. I THINK I solved it by turning Bluetooth off on the kid iPad but at this rate who know.

So what's a solution here? Is this simply the side-effect of repurposing my Apple ID? Breaking the Apple annoying "one device for one person" model? Simply put: is there any way to "disconnect" one Apple Device from the HomePod ecosystem, prevent it from playing on a HomePod?

While he's getting to the age where I will get him a proper AppleID so it will be truly his device, I worry that this problem will perpetuate with family sharing. I appreciate that Apple is trying to create an easy connected ecosystem, but I wish more control was put in the hands of the users so we have a bit more say over the individual device level of participation in that ecosystem. Not all devices are used the same way in that ecosystem.

Thanks Community!