Well, after a lot of fooling around with different things, I take back what I said previously. The problem most definitely IS the CAC.
When Apple removed CAC support in 10.7, it looks like they did it by just removing the libraries needed to access the CAC, but left the operating system hooks in place. Prior to 10.7, if you had fully enabed CAC on your system, you'd use your CAC/pin to do things like unlock system preferences. Well, the code for that is still there, they just removed CAC support.
So if you install third-party CAC support - Thursby PKCard, CAC_KEY, or any of those other things, or if you still have the libraries from Snow Leopard, once you put in and access your CAC (by using it on a website or signing an email) the system wants to use it to authenticate other things too (like system prefs). Unfortunately those third party solutions don't support the system hooks, so it doesn't work.
What's worked for me recently is firing up activity monitor and force-quitting securityd and the processes that my CAC software spawns (I use cac-key). Once I do that, I'm usually able to get my prefs unlocked without rebooting.
Buying and installing something like Thursby's AdmitMac PKI should solve this, since it's designed to support the entire system from login through active directory integration. But it's a bit pricey at $200....