All games, or one in particular? It sounds like the pad thinks it is busy doing other things. Try shutting aveything down. Radios, other apps, bluetooth. anything you can find - to take the load off the processor. Then, when it seems ok, begin to introduce that other stuff until you see what is causing the issue.
if you really want to run it down, there is a nifty app called Status. One of those screens displays CPU activity. At idle, I run between 5-10% CPU usage, and you should too. If it is running higher, then use that display to turn stuff off and on to see what the culprit is.
If this is happening on one specific app / game - check with the developer and see what he has to say