It's either or. The Mac will not let you use the speakers and another sound output device at the same time. Since the port is occupied, as far as the Mac is concerned the speakers are being used.
Your only hope is to get that remnant out of the headphone port.
By the way, have you checked System Profiler? (Apple Menu / About This Mac / More Info).
What does the Mac says in Hardware / Audio?