It seems one of the mics for noise cancellation is cancelling your own voice, because the proximity sensor is believing you are close, while you are not, and your own voice is getting fed through the top one.
Do this test:
put the phone screen right in front of your mouth, 4-5 inches from it, and speak to the center of the screen. Not to the mic, not to the top speaker.
Mine just mutes me completelly.
It should (or so it should have been designed) NOT cancell noise when NOT against the face. When against the face, the sound entering through the top part SHOULD be considered not-desired, while the sound entering through the bottom mic should be considered as the desired source. BUT when you take your phone away a few inches, e.g. to read the agenda out loud to your other person, both mics are EXPECTED to have similar input, so you CAN'T just cancell them out.
The proximity sensor should take care of that, and in mine it's not.
It gets my voice correctly when I put it on "speaker" mode, though. (you manually told your phone you will be away from the phone), so that part works.
I'm taking it to the AppleStore right away. It could have something to do with the rattling noise described in other posts.