Iphone 4s shows charging symbol but does not charge?

I started having an issue a few weeks ago where my phone kept turning itself off at around 40% charge, most often when I was using the camera. Trying to turn it back on again it would show the empty battery symbol and to connect to a power source. After leaving it for around ten minutes I could turn it back on and it would be at the previous 40% or somewhere thereabouts. It was annoying but for the most part I let it go.

Last week or so I charged the phone overnight and looked to find that it was only on 35% or so after being left on all night. Naturally I assumed I didn't plug it in right and then left it in for the afternoon, where it still showed as being connected but reduced down to around 15%. Again I checked it was plugged in right and left it. The battery eventually went empty and upon leaving the cable in, the phone would turn on every half an hour or so, realise it had no battery and then turn off again. Then it eventually just stopped responding altogether.

My phone is out of warranty, having bought it in January 2012, and I'm currently backpacking through rural Australia so given that I have no money and the nearest Apply shop is a good few thousand kilometres away going to apple wasn't an option, so I ordered a replacement battery thinking this was the issue. Having just installed the replacement battery, the phone turns on and works fine, so I plugged it in and once again it is showing the charging symbol up the top right hand corner and on the lock screen, but the battery is still decreasing in percentage so I have simply turned it off.

Now, in the connector port of my iPhone there is a bit of fluff or something that has always made the connector plug in at an angle, but it has never been an issue and I don't want to go sticking things up there in case I damage the pins, plus if it is registering as charging then I don't see how that could be the problem, as it hasn't been fpr the past year.

The iphone registers in iTunes and Im currently downloading 6.1.3 again to restore to factory settings but I still don't feel this will resolve the issue - I'm only doing it for the sake of being thorough. I am pretty positive it's a hardware issue, I just don't know what to do about it or if this is a known problem, are there any means I can go through to get my phone working again?