Something has gone wrong, on an iPhone 4 the erase all operation should be almost instantaneous.
Turn your phone off: press the Sleep/Wake button & Home button at the same time, keep pressing until your phone turns off, then release.
Then, leave the USB cable connected to your computer, but NOT your phone, iTunes running, press & hold the home button while connecting the USB cable to your dock connector, continue holding the home button until you see “Connect to iTunes” on the screen. You may now release the home button. iTunes should now display that it has detected your phone in recovery mode, if not quit and reopen iTunes. If you still don’t see the recovery message repeat these steps again. iTunes will give you the option to restore from a backup or set up as new. In your case, select "new".