Bootable external drive will not accept FileVault password on boot, but unlocks when mounted (Sequoia)

I am trying to create an external SSD drive running Sequoia and enabling FileVault. I have done this in the past with older OSes with no problems.

In this case, I can boot off the drive without issue until I enable FV. When I attempt to boot off the drive with FV enabled, it shows me a list of users but does not accept the correct password. I have also tried using the recovery key. It shakes it off as if it is bad. However, when I boot up on the main drive, I am able to use the same password to unlock the drive.

I have tried several times on 2 computers with the same result.

Hardware is a T2 Mac mini (2018) running Sequoia. External boot is enabled in firmware. Drive is formatted with APFS.

Any ideas?