Big Sur - Mouse bug - cursor and pointer missmatch

I updated my mac to Big Sur a few days ago, and after I connected my external display (Dell U2718Q scaled to 2560x1440) and my Magic Mouse, I had issues that the mouse cursor sometimes itself does not represent the pointer.

When I press the mouse button, the click itself happens somewhere else on the screen. The cursor/pointer missmatch is consistent as long as I move arround on the screen. So if it's 100px to the left and 200px to the top, it will always be like that, until I move the mouse once to the top left corner of the screen, than it's back in sync.

I have already restarted the Mac and tried the NVRAM reset. and already deleted and reconnected the Magic Mouse.

I can not fully confirm yet, but pretty sure the same issue happens if i just use the integrated trackpad.

When I used the mac before, without display and magic mouse, there was no problem with the cursor.

Any ideas would be appreciated.