Keychain sync issues after adding smart card in macOS Sonoma 14.2.1

A few days ago I decided to add a smart card (Yubikey USB-C model) so that I can use a simple PIN to sign in to my MacBook Pro when it is plugged in. I had no problems adding or using the Yubikey, but immediately noticed 3 main symptoms: 1) My iCloud Keychain passwords were no longer on my MacBook, 2) My Home accessories all showed as Not Responsive (even though they were responding on my iOS devices), and 3) my password was intermittently not accepted when attempting to unlock Notes.

I was due for a clean install so I erased my MacBook and reinstalled from a Time Machine backup. The problem has persisted post-install. A restart fixes the issue temporarily but the passwords end up disappearing and my password stops being accepted in Notes.

I have tried to sign out of my Apple ID and sign back in. I've also disabled the Keychain from iCloud settings and re-enabled it to no avail.

Has anyone else out there seen a similar issue and found a solution? I know I can likely do a clean install on my MacBook but that seems like overkill for an option that I imagine is something simple I'm overlooking.