Uncommanded Auto zoom on Browser

Hey guys,

Ever since I upgraded to Big Sur (11.1), my apple magic mouse seems to randomly zoom in on my browser window. I cannot zoom out using the ctrl-scroll action.

It's really annoying as I either have to scroll sideways or close the window (refresh of the page doesn't reset the zoom).

Any idea how I can stop that from happening? It usually happens when I touch the scroll surface on the magic mouse. I think it doesn't happen on mice with a regular scroll wheel. It also only happens in Chrome. It doesn't appear to occur on other applications.