Locked EFI does not respond to keyboard or mouse input

I recently secured my Mac Pros with an EFI password, however when I start the machine with CMD + R for Lion Recovery Mode, I am prompted with the password screen, however the mouse nor keyboard respond to input. I have tried with Apple Bluetooth Keyboard/Magic Mouse as well as an old Apple USB keyboard and mouse. Neither work.

This is annoying, what can be done to fix this?