How do you disable Magic Mouse Zooming via swiping (scrolling)?

On some web based apps that have graphics, in Safari, when you're in the image window and you swipe the scroll area of the Magic Mouse, rather than the web page scrolling, the image zooms. It's hard not to brush the scroll function when you're just trying to get to the select button, so the images end up zooming when you don't want them to. I'd like to disable the scroll function of the mouse altogether. How do I do that?

I'm using Mavericks 10.9.3 and a Magic Mouse