speaker keeps switching to iphone

I have my iPhone 14 Pro (Just updated to 16.2) setup by default, so that listening to incoming calls is routed through the iPhone's speaker (not bluetooth). So it's handsfree. Set it up in Settings>Accessibility>Touch>Call Audio Routing.

But periodically, and randomly, it would switch back to the iPhones earpiece. I would have to choose Speaker again on the call screen. This would take, and stay on speaker. But sometimes, like today, it would just keep switching. No matter how many times I keep tapping on speaker. This was happening even before I updated to 16.2. This was also happening on my previous iPhone 11 Pro (since I got it back in 2017), running up to 15.3.1. So it seems like this is an iOS issue, that has migrated in every updated.

Note as well, whenever I tap on the Speaker icon on the call screen, a drop down menu appears. Choices are iPhone, Speaker, and Bluetooth. I have to keep tapping on Speaker. I recall from previous iPhones (pre 11), you only had to tap the Speaker icon. And it just turns white for On, and not white for Off. No drop down menu. No issues when there was no drop down menu.

Why is this happening, and is there a fix or workaround, so that my phone stays on Speaker by default? As it's supposed to.