Magic mouse disappears when using the link keyboard and mouse between two Mac's

I use the link keyboard and mouse feature between my two Mac book Pro's daily and multiple times during the day the connection will drop so I have to manually reset the connection or wait for it automatically reconnect. In some cases, the magic mouse connected to my primary Mac book completely disappears from both devices forcing me to turn the power off and back on.

This only happens while working on the secondary Mac book. Its something that started after an OS update a few months back and has persisted even after a recent upgrade of one of the Mac books from M1 to M3.

Both devices are connected to the same network, placed on the same desk next to each other and the AP is on the same desk as well.