Speaker phone automatically turned on whenever I answer a call?

Dear community,

Whenever I answer a call, its speaker phone is automatically turned on. How can I turn this feature off? It seems to be a software issue, because when the I got my iPhone 4S brand new, it didn't have the problem. After restoring from my iPhone 4S, the problem I already had recurred on the iPhone 4S as well. Any suggestions are greatly appreciated.

Thanks!

88kin