Headphones only work wierdly?

This is a question on whether the headphones are at fault or the headphone socket. Within the last 2 months, I had to fiddle with a couple of headphones that have been plugged in and worked perfectly, but suddenly they can only work on one side of the speakers on the headphone set unless I fiddle with the angle and sometime pressure on the jack to make both side work and yet I don't know if its the headphones or the iPod doing this as this is annoying me when I try to listen to any of my podcasts or audiobooks.