iPhone 8 speaker button

When I make a call, I usually use the speaker. But when I tap the speaker button, I have three options appear: iPhone, Speaker and my iPad.

I want it to go straight to the speaker. I restarted my iPhone, which seem to fix it, but those options appears again a few days later.

I do have FaceTime on my iPad, that might have something to do with it.

It has happened previously on my old iPhone also (iPhone SE 1st Gen).

It happens on WiFi. WiFi calling is turned OFF and "calls on other devices" is also OFF.

I want to know if there is a permanent fix?

Current iPhone: iPhone 8 with iOS 13.4.1.

Thank you.