I ended up uninstalling and reinstalling iTunes using the order specified in Apple's support document. That did not fix the crashes I was getting when syncing media with my iPad 2. I ended up calling Apple and they had me do a complete restore of my iPad 2 and setting it up as a new user. At that point I successfully synced all my data (except photos which I didn't try). I then restored the backup data and settings. After that iTunes immediately tried to sync and made it all the way to the "optimizing photos" stage where it promptly crashed. I tried removing the iPod Photo cache folder, but that just caused iTunes to try and re-optimize all my photos and it ended up crashing at about 2450 out of 1600.
The only way I could get iTunes to sync without crashing was to delete the new photos I added and then sync one album at a time.
I'm still noticing much higher than normal CPU usage when syncing music and podcasts since restoring my iPad backup data, though I haven't seen it crash on music or podcasts yet. I have a feeling it will crash again if I try to add a new photo.
At this point, it looks like the "fix" is to either do a restore on my iPad 2 and not restore from the backup data (not something I want to do) as there is apparently a setting on my iPad 2 that iTunes 10.4 does not like or revert back to iTunes 10.3.1 (not sure if that will work or not).
Either way, iTunes 10.4.0.80 is garbage since I never had any problems syncing under 10.3.1 or earlier.
For those interested, to downgrade back to iTunes 10.3.1, you need to either have saved the install file for it or find a copy of it out on the net.
At that point you can follow the instructions for downgrading on this page (even though it's for iTunes 10.5 beta, the steps should be the same.