Smartcard required authentication prompt not asking for pin

I followed the directions on the Yubico and Apple sites to set up my yubikey for smart card required authentication. I'm still working on building out my MDM solution with the smart card, only payload so to enforce it now while I'm still working on that, I utilized the below code.

sudo defaults write /Library/Preferences/com.apple.security.smartcard enforceSmartCard -bool true

I'm able to login with my smart card through file vault without any problems. I've also managed to add the functionality of additional smart card requirements for sudo and su in terminal.

I found a great resource by synthetic-intelligence and that complied the Mac support pages onto a Github Gist and Forked it so that I could add the Yubico section and utilize it while building out my payload and mdm.

Most all auth prompts have switched over to requiring pin however, there are a few lingering that say password still. Such as when you go to System Settings> Users and Groups and attempt to change the password for the user there the pop-up comes up requiring a password, not pin. But when you type the password in it says smart card required. I also get the pop-up requesting password instead of pin when attempting to add a helper and VS code.

I can't figure out how to get these last lingering password prompts to switch over to pin for authentication.

Thanks to anyone who can help.