Can't talk and listen in phone calls but speaker works

My phone rings and plays the ringtone when I receive incoming calls, and it also plays music perfectly well. But people can't listen to what I say neither could I listen to the the caller. I tried restoring the phone, but it still doesn't work. Is it possible because I have an ipone 4 and I updated to IOS 6, so it messed up with my phone since it's not an iphone 5? What happened to my phone? Is there anyway that I can try to fix it? Thank you.