I was also stuck in Restore recovery mode... I finally upgraded my iphone4 successfully. Here is what I did and I hope that it works for you...
I couldn't get the upgrade to work from my home pc which is where I normally sync my iphone. I installed a fresh copy of iTunes 10.5 on my laptop. I copied the ios5 update firmware file from my home pc to the laptop. I followed Ichiboi's instructions below to install. Once I had the phone upgraded to the ios5 I tried to restore my backup file from the home pc to get my contacts, apps, and the rest of my files loaded back onto my phone. This didn't work either so I copied all the backup files to my laptop also and restored from there. It took a couple of tries but finally everything went OK. I only got a few of my apps loaded back up so I had to manually reinstall some of my larger apps. See how to copy my backup files to the laptop below Ichiboi's stuff...
I then connected my iphone with the new ios5 to my home pc and re-synced. It thought I was still in recovery mode and tried to load the failed backup. I just hit the cancel button and continued on with setting up my phone. Once I had my phone back to normal (you have to reload you apps and recreate all the folders and layout for the app icons) I then deleted the failed recovery files and created a new backup.
My phone works great so far with all the new ios 5 options. hope this helps others too!
LFNNS
=======================
((From ICHIBOI on the bottom of page 7 of this thread))
I have successfully updated my iPhone to IOS 5.
Be sure you have iTunes 10.5 installed
Launch iTunes 10.5 and then…
Mac users: hold down the OPTION key then click on “Restore”, locate the aforementioned IPSW file
Windows users: hold down the SHIFT key and then click on “Restore”, point iTunes to the downloaded IPSW file.
Firmware files have an .ipsw extension and can be found at the following locations:
On Windows XP :
Documents and Settings\<User>\Application Data\Apple Computer\iTunes\iPhone Software Updates
On Windows Vista/Windows 7:
Users\<User>\AppData\Roaming\Apple Computer\iTunes\iPhone Software Updates
On Mac:
~/Library/iTunes/iPhone Software Updates
I thought I have broken my iPhone but now I'm glad it worked.
Good Luck!
========================
How to copy your backup files to a new computer. Locate the backup files in the following directory and then transfer to the new laptop where you have iTunes installed.
C:\Documents and Settings\USERNAME\Application Data\Apple Computer\MobileSync\Backup
=========================