Why does my cursor freeze with both magic trackpad and mouse?

Sometimes when I am using my computer, the cursor will stop moving in response to the mouse (or trackpad, doesn't seem to make a difference). The ability to zoom and click remains, but the cursor will not move until another window is brought to the front (e.g., with command-tab). This can be incredibly frustrating if I'm using a program that does not allow switching to a new window without quitting. Rebooting the mouse/trackpad does not solve the problem. Any ideas?