Magic mice are sometimes unresponsive on my Mac studio

I have both battery-operated and rechargeable magic mice. Sometimes, I click on something, but it won't open. This is very frustrating and happens with both mice. Resolving it usually requires a reboot. Has anyone experienced this with their Mac studio? I have the iMac Pro display XDR and assume it's unrelated to the monitor.