iPhone 4 stuck on iTunes with plug screen

I have an old iPhone 4 (32 gigs) updated with iOS 7. The phone hasn't held a charge for many months unless it's plugged in, the charging lightening bolt doesn't appear when it's plugged in, and the power-off slide bar doesn't appear when I press the home and power buttons. Also, iTunes on my computer does not recognize the phone when it's plugged in. A few days ago, the iTunes logo with the plug image suddenly appeared on the screen and I cannot get it to go away or get past it to access my phone. When I have been able to turn the phone off and then back on, the apple appears, but eventually the blue iTunes logo with plug comes back and stays on constantly.

I have a recent iCloud back up (since iTunes hasn't recognized the phone when I plug it into my computer), but not sure what good that will do since I can't get past the iTunes logo and my computer doesn't recognize the device. I have a new iPhone 6, but have important information stored on the old iPhone 4 that I didn't want transferred to the new phone. Any ideas on how to get into the phone again? Any ideas are much appreciated. Thank you.