Even after the two failed migrations, the server kinda functioned and kinda didnt. For instance, DNS and VPN worked fine. But Radius threw errors all over the log and, as a result, all of my AirPort APs stopped authenticating users. There's no radius pane in the Server.app and the Airports did not show up on the side by as is suggested by Apple's documentation. So, again... how does one troubleshoot? Am I missing some basic premise about this incarnation of OSX Server?
i had an old server running 10.6.8. there were really only two things i wanted to move across in the migration: email, and collaboration (wiki) content.
sounds like i tried something very similar to what you tried. created a new fresh virgin 10.8 install on one partition with my old 10.6.8 on a separate partition. booted 10.8 and started server app and tried the migration.
i got the first warning that you mentioned about open directory errors. however, when i looked at the set of users and groups on the new server, it seemed to match that of the old server, so i was hoping things were ok.
however, of the two key services that i wanted to transition, first email failed. it seemed to be configured correctly and it seemed to be using my SSL cert, but i could not get clients to connect. i'm not a dovecot expert, so in the hour or so i spent poking around, i couldn't come up with any simple explanation for why it wasn't working. the server app (and the server commandline interface (serverinfo) seemed to think everything was fine, but i could still not connect.
second, collaboration (wiki). the server app and the commandline (serverinfo) again seemed to think everything was fine, but the service simply didn't come up. that is, if i clicked the link in the server app to access the wiki, it did not work.
overall, i found it very strange - these things were oddly simultaneously happening:
a) what i was trying to do was very straightforward and "vanilla" only involving a few of the core services - nothing exotic.
b) it seemed to fail totally and completely on all the important services, yet
c) the simplistic diagnostic/monitoring tools provided told me that everything was ok.
overall, i found this very disturbing and it makes me question whether a) i am doing something really dumb or really fundamentally wrong and not realizing it or b) perhaps it is time to give up on mac os x server.