I've resoled this problem with my 21.5" iMac 2017 with macOS Big Sur 11.2.3
In my case, this had almost nothing to do with the mouse, the mouse was just the most visible effect of the root problem.
I started watching my activity monitor like a hawk, I noticed something was taking up nearly 30%- 40% of my processor all the time, even with no programs open. It was the Window Server. I started digging into what could cause the Window Server to eat up so many resources. I have two external 1080p monitors hooked up to my iMac via USB-C, the primary iMac display was set to be scaled so items would be roughly the same size across the 3 displays. Well if you go into your Display scale settings, adjusting to anything except "Default" causes a new notice to appear: "Using a scaled resolution my affect performance".
So the Window Server was bogging down EVERYTHING to scale my primary display and push two external displays, and the Bluetooth mouse performance just happens to be one of the first things to go when your system is hungry for more CPU.
I've been running 3 displays on this iMac since at least Mojave without issues, I'm not sure what "advancements" Apple made to cause this to happen but I'm happy to have a working computer again.
Hope this helps!