1 DualSense switching between 2 Apple TVs

I have two Apple TVs at home (upstairs and downstairs). I bought one PS5 DualSense Controller and paired it with both Apple TVs. I only use one AppleTV at any time.

It’s a pain switching Apple TVs for my PS5 DualSense Controller. Unless I’m aware of an easier way, the only way to switch is to Forget from both Apple TVs, and then re-add to the one I want to play on.

FWIW - I tested with the SteelSeries Nimbus+ and it behaves much better. Connecting at one Apple TV automatically disconnects it from the other Apple TV.

Any thoughts on how to have the same easy switching behavior with the PS5 DualSense Controller? Or, if there’s an easier way to do it?