I noticed a few different things right away. I was able to create folders, e-mail program was updated (could read mail from all my accounts either by account or in one folder), and camera zoom.
I was disappointed that there is no background image behind the icons, just the unlock screen as usual. I'm not sure of any speed increase/decrease, but memory usage is higher. I'm sure glad that I have a app that frees memory.
To help the upgrade go faster, I whipped my iPhone clean of all my music. All my music is backed up and I've been meaning to "clean house" anyway. All that iTunes had to back up is the apps and other data like contacts. This reduced my upgrade time to about 40 minutes.
Since I pre-ordered the iPhone 4 my upgrade on my 3g was more out of curiosity than anything. If I wasn't getting a new phone so soon I would have waited a bit longer. I do like the app folder feature a lot though. That in itself makes iOS4 worthy on the 3g. I didn't like all the "screen flipping" to get to apps. Although my apps were organized, it was a bit annoying.