If you are seeing the box that says Headphones, then the device thinks the headphones are still connected. This could be caused by dirt, lint, or other debris in the headphone port. You can try inserting and removing the headphones several times quickly and see if that helps. If not, use a strong flashlight and magnifying glass to see if you can see anything in the port. Also, try powering the device off and back on. If that does not help, then try a reset. This will not affect any data. Hold the sleep/wake and home buttons together until you see the Apple logo and then release. The phone will reboot. Test after that. If that does not do it, make an appointment at the Genius Bar of the nearest Apple Store to have the device examined by Apple.