Iphone 4 software upgrade issues

My Iphone 4 32Gb software upgrade is always not a smooth process. From 4.0.1. to 4.0.2 & 4.0.2 to 4.1, i face the similiar issue and each time took me about 3-4 hours to get it completed. It always failed during the initial upgrade process,then it prompt to restore to apple default and update. Then it will keep failing the restored verification process for about 5-8 times and by chances it managed to get it success upgraded to the higher version.Then I had to restored the backup from itunes, but it seems that the folders will be messed up and i need to redo the folder again. Is this common process for software upgrade in iphone? If not, can i consider is this phone has some hardware problem on the upgrade potion? As i do not see any connections issue except the two upgrade i apply to the phone.