Same issue here...
iPod Touch Gen2 16GB, 2 Minute till sleep, 1 Hr Push settings, Bluetooth off, and brightness manually set to about the 30% mark.
Before iOS 4.0 I could go several days without a battery charge. It now always has a dead battery in the morning when I wake up. With the previous OS, I always left wifi on before and did not have problems. Now you can almost see the battery draining.
Other than that, the only problem I have had with 4.0 is sometimes there is no audio (no music, no game sounds, no "clicks", nothing...) when it wakes from sleep even if the volume bar shows it is turned up.