Actually, that is something else that I have experience in. My son had a Lifeproof case on his phone and when he called me the sound was awful. He called Lifeproof and they were aware of this issue and sent him a replacement case immediately. It seems that it has something to do with both the noise cancellation opening by the camera, and the membrane by the microphone. Give them a call, they are real good about it. They will send you a replacement and just ask you to return the old one.
If you want to test this before you call Lifeproof, you can remove the case and do a test call. My son did the same you did, used the Bluetooth and it worked great, then removed the case and the calls were crystal clear. Good luck.