Voice quality issues - when using iPhone 4 - anyone else having issues?

I am experiencing voice quality issues when holding the iPhone 4 as a normal phone to my head. I have made test calls to my home line VM and can tell a big difference in my speech quality when I use it in other configurations (i.e. speaker phone or with a headset/headphones. When I talk directly into the microphone my voice sounds muffled and choppy. I feel this is an issue with MIC location or MIC volume which is fixed and cannot be adjusted. Apple replaced my iPhone 4 on Wednesday and it has not helped the issue. I can hear the called party voice fine, but several folks have noted that they have issues with my voice. I would like to know if anyone else is experiencing this or have any input to help resolve the issue?