Upgrade iTunes on the Mac and the OS if required so that you have the correct version for syncing with iOS6.
Alternatively, you can copy the ENTIRE iTunes folder from the Mac to the PC and then you should be able to sync to the PC without issues.
You'll still need to create a backup on the PC to restore back to the iPhone5.
You can also try backing up the 3GS to iCloud and restoring the iPhone5 from that backup, but that will not restore or sync any media.
Sorry it is going to be a few days before I get back to this. I was able to recover most of the functionality using the iCloud backup which worked great. But I still need to sync to the computer for greater control.
So, are the mac itunes files the same format as the windows itunes format, are the files swapable? I am looking forward to trying this.