This could be due to some debris or dirt inside the headphones jack that makes the iPhone think that the headphones are connected even when they are not.
Try to see if you can clean the jack being very careful, with canned air as sold in electronics store. Some people have been successful by introducing and withdrawing the headphones jack several times in a row to displace whatever is in there if it is lodged in.
Avoid introducing anything else like a pointed object. You could damage the jack.
If all of this does not work, I would try first resetting the phone by holding the home button and the power button together for about 15 seconds or until the whilte apple appears, then waiting until the phone resets. If the problem continues after this, take it to an Apple Store to see what they believe should be done.