I can't explain why this works, but I was able to solve my issue.
As explained above, I am, and have been, booting form the external SSD for years with no issues. Soon after I set up my external SSD, years ago, I deleted all the files from the internal Mac HD and used that disk for non critical storage.
Upgrading my external SSD to Big Sur exposed some problem and I think it is related to the internal HD, even though I had nothing critical on that drive. Maybe there was a recovery partition on that drive I didn't know about.
After I upgraded to Big Sur, if I had my Time Machine drive attached the computer would boot to recovery mode. After trying a bunch of potential fixes, including repairing drives, the computer would start up with the picture of the folder with the question mark and sometimes finish booting.
Last night, I decided to install Big Sur on my internal HD (which did not have an OS prior to the upgrade) even though I am not booting from it. I guess the install put information on that disk that makes the computer happy and it now boots to the external SSD every time regardless of what other additional external drives are attached.
This worked for me and I hope it helps you.