Hi John! I'm glad I found your post. I have a new Quad i7 Mac Mini "Server" and have been struggling with it the past couple days. I don't need the "Server" parts, but rather bought this version for the processors it came with.
Over the past couple days I found that it's not possible to restore to Lion (Desktop); it restores to Lion (Server). And while the Server version is supposed to be identical to Desktop, a call to AppleCare confirmed my troubles - that "Server" _may_ have slight differences here and there that can cause compatibility issues.
I had a heck of a time trying to install CS5, in fact I just couldn't, and hours of Adobe chats and Googling couldn't help me. I even tried to install Lion over TDM to my Mini from the AppStore image, and it wouldn't boot that way either! It seems the Server edition of the mini only like Lion Server...something keeps it from booting from a clean Lion (Desktop) install or just the Installer itself (on a USB drive I image'd).
Anyway, I managed to get Snow Leopard onto the Mini and am typing this now on it...while CS5 installs without a hitch (so far) in the background(!). My process wasn;t quite as smooth as yours seemed to be, however.
I was using a retail 10.6.3 disc and the recovery discs from a mid-2010 MBP (shipped with 10.6.3). I installed from the retail disc onto the Mini in TDM, then it rebooted (I didn't swap in the recovery disc since they were the same version). On reboot, the installer popped up and said "OS X Can't boot from this disc" and the install was screwed at that point. I guess I should have tried putting in the recovery disc nonetheless?
Anyway, I also had the retail disc on a USB drive, booted my MBP from it, then installed onto the Mini in TDM. This time it seemed to work and the MBP rebooted successfully and I went through all the setup screens, etc. I then ran Software Update and got to 10.6.8 and shut down.
On reboot of the mini with fingers crossed, it booted! I was ecstatic. But the first thing I noticed was that my dual display setup wasn't working right: the display hooked up to the Thunderbolt port was not being recognized. I disconnected it and rebooted and all seems OK now. I ran SU again, but it didn't fix my Thunderbolt-attached display, in fact System profiler shows two identical displays attached, but the Thunderbolt one is a different model display.
My current plan is to install the Lion-problematic CS5, then clone it, then try an "upgrade" to Lion (Desktop) and see if I can boot the mini into Lion (Desktop). If it works, and my Thunderbolt display works, etc, I think I can live with Lion (although i understand what you mean about how annoying it is!).
Otherwise if it won't boot, I will try reinstalling SL the correct way and swap in my recovery disc this time (BTW, how do you eject the disc when the selector screen comes up? And once you do the swap, do you then select the recovery disc and it will continue the installation process?), and hopefully that will fix my Thunderbolt display issue and I'll happily live with 10.6.8!
Thanks again for your hard work!