Bluetooth devices often got disconnected from Mac mini with M1 chip.

Poorly shielded High Speed USB external Cables, Drives and Devices can cause Wi-Fi and Bluetooth issues.

see > https://support.apple.com/en-us/HT201542

First try rearranging your setup so that all of the High Speed USB cables, drives and devices are in back of and to one side of the Mac Mini. Then if the Bluetooth does not improve, try disconnecting one cable at a time from the Mac Mini to see if that helps.

Bluetooth issues are common for many users of the Mac Mini. This is usually caused by interference from other hardware and/or poor cable shielding. Try to keep USB components away from the Mac as far as possible and try to move some of your cables around and see if this will fix the issue. I doubt 11.6 will fix this issue as Apple stated they addressed this with 11.3 but I see some still have issues. I have no bluetooth issues with my Mac Mini M1 running 11.5.2..

I have had issues both with Magic Mouse 2 and Magic Keyboard. My keyboard would have trouble registering characters on an application. Would have to disconnect and re-connect or switch to using a cable to connect keyboard. Magic Mouse has tracking issues, cursor would move erratically or not at all even fully charged. Bluetooth remains an issue on M1 Mac mini. I question the interference as being the problem but I guess it is a possibility. I used an Intel Mac mini with the same setup in same room with same Keyboard and mouse and never had any Bluetooth issues. This is why I tend to dismiss the interference as the problem. It could be a poor quality Bluetooth chip in Mac mini or its simply poorly shielded. I am considering going back to both wired keyboard and mouse for peace of mind.

Wired gives peace of mind, no doubt about it! The Magic 2 keyboard functions as both wired and wireless, so all you need to do is swap your mouse for wired. Can recommend the Magic 2 Trackpad. All the best!

Latest Big Sur is 11.6. See if it improves?

Switch to wired, be happy.

Does it improve with the speaker powered off? What make-model Monitor? How is it connected?