So, I solve this issue and one more. I really can't believe nor understand why Apple is unable to create a server.app that can handle the simpelst things. What trainees are working on it?
You need two fixes:
1. http://support.apple.com/kb/TS4539 (OS X Server: Saving a certificate identity to the system keychain does not work with Server services)
The importan part is, that you need (!!) to reboot your machine. Only than the files are generated. I don't understand what's so hard to generate them right when adding the certificate but that's an other story.
If you than try to use your now finally available certificate, you are hit by this problem:
2. https://discussions.apple.com/thread/4266473?start=0&tstart=0 (Strange issue with Dovecot (err 89))
And yes, that's because server.app comments out the SSL stuff in the dovecot config. Which of course makes a lot of sense if you want to use SSL. The bad sideeffect is, that dovecot won't startup with this wrong config.
Enough hours lost by this Apple botch...