Mail keeps asking for PIN with CAC

I digitally sign emails with a DoD CAC (Common Access Card) that's authenticated with a PIN. On several machines (primarily my 2021 14" MacBook Pro), on every version of MacOS 12 (I'm currently using 12.4), a popup window appears every minute or so asking for my PIN. On every previous version of MacOS, I would only need to enter my PIN when I sent the email. This is very aggravating because I'll be typing away at an email and then those keystrokes get captured in the popup window, then once I enter my PIN, it loses my place in the email.

Is there a fix or workaround so that I only need to enter my PIN once?