Well... now I can not be absolutely sure if it was exactly this thing, that might helped, but everyone can try it out to see.
1. Open Keychain Access
2. Go to Edit -> Change Settings for Keychain "login"...
3. Activate both options available in this window ("lock after xx min inactivity" and "lock when sleeping").
These options are probably both inactive by default - I have not touched these ever before and they were both unchecked.
Now after this I do not have any problems when using my ID card. Logout and re-login or even restart might be in place after that - at least i made the restart, but for other reasons.
Please try it out - then we can see, if it solves the problem for you or not.
P.S. This thing will have one side-effect in daily use - when any application tries to access the keychain after this timeout or after sleep, you will be prompted to enter the login keychain password (same as your login password by default).It might appear as annoyance at first, but actually it offers additional layer of security - and this timeout value can be adjusted to suit your needs.