I'm another 3G user who upgraded to iOS 4 and 4.0.1 and have regretted doing so for the last few weeks.
Some of the issues that I've been experiencing are dropped calls, unable to answer incomming calls (can't swipe the answer button!), very, very slow performance using apps, both iOS 4-ready apps, and those that are provided by Apple.
I've done clean installations (not upgrade), set up the phone as "new" in iTunes, used a Mac and PC to do the restore, etc., but still continue to have issues.
The only way that I found to downgrade to 3.1.3 was to use jailbreaking software and doing so. I didn't like that because some apps wouldn't work under the jailbroken phone (Apple Remote), the apps would start up, and shut off.
I upgraded again to 4.0.1 because needed my phone, but will look at downgrading options again.
I want a solution provided by Apple to downgrade to 3.1.3 - not Googling and using third party software to do so. Apple should allow us to downgrade without users needing to do "hacks" on their phones!
The amount of signal bars has also been affected. I get one or two bars indoors, while my wife - still on 3.1.2, gets full bars. I experience dropped calls like never before, she does not.
This is clearly an OS issue! Problems that have risen that were not there before, which can only be attributed to the software upgrade.
Apple, FIX IT! I already submitted feedback, and hope to hear from Apple soon.