The only thing I have found to work is to log out of Game Center through settings, and not log back in. The only way I have been able to do it is fallowing these steps.
Close everything running.
Preform a hard reset by holding the power/sleep button and home button. Hold the buttons down until you get the Apple logo.
As soon as the Apple logo shows, release the buttons.
Once the device has finished booting, immediately open settings.
Scroll down to Game Center, and tap on it to open it.
I usually have to wait a couple minutes, however it usually loads up. As long as you get the loading indicator, you just have to wait.
From in there, tap your login E-Mail address, and select log out.
From there, just don't log back into Game Center.
Yes, you loose all the benefits and features of Game Center, however unless the game requires it, they will load after about a minute.