iphone XS speaker phone doesn't work

Lately when I answer a phone call by the speaker phone, the speaker volume became very low. I hardly can hear the caller's talking, I have to put the phone next to my ear to hear it, just like the speaker is not on. But when I playing music using the speaker, it works fine and sounds normal, it only happens when I switch the phone call to the speaker phone. How can I fix this problem?