How do I stop mac from booting to recovery mode if I take too long to login

If I turn my Mac on and then go away to get coffee or get side tracked before I login, a message will appear below my login asking me if I forgot my password and offer to help. I can enter my password OK but the next time it boots it goes into recovery mode and takes forever to boot. I've got FileVault installed so it'll bring the login screen up pretty quickly and the login seems to go into "help" mode pretty quick as well.

It doesn't matter how many times I reboot - it will always goto the recovery mode boot. I can stop this by turning the Mac off and then back on while holding down command-option-p-r. But it shouldn't boot me into recovery after I entered the correct password and force me to hard reset.

Anyone know how to turn this "feature" off? It's pretty annoying....