Bryan,
This may help you regarding the 3rd party cert issues.
I was having problems with a legit GoDaddy cert I was trying to use for a web site in Server.app 4 (Yosemite) but I believe this also applies to Mavericks Server.app 3:
I found that the Web Services Site creation panel would always default to port 80 when I chose my cert. It is a good and valid cert and it trusted and intermediate certs are installed.
But when I chose an Apple default cert it would get the proper port 443.
If I tried using my GoDaddy cert and changed the port to 443 and tried to save I would get the message:
"Port 443 can't be used without an SSL certificate"
"You must choose an SSL certificate to use port 443. If you don't want to choose an SSL certificate you must use a different port"
But the cert looks perfect KeyChain Access.
Although Keychain Access would not show the problem, the problem COULD be detected by examing /etc/certificates where I found that unlike the Apple default certs, my GoDaddy cert was missing the fourth member of its set (the private key one ending in ".key.pem")
1) mysite.example.com.CAGobbledygooknumbersandletters.cert.pem
2) mysite.example.com.CAGobbledygooknumbersandletters.chain.pem
3) mysite.example.com.CAGobbledygooknumbersandletters.concat.pem
4) mysite.example.com.CAGobbledygooknumbersandletters.key.pem THIS WAS MISSING
Finally, I found this Apple tech note which resolved the problem:
http://support.apple.com/en-ca/HT203731
OS X Server: Access Controls might prevent a certificate identity from working with Server services - Apple Support
After using the Access Control fix listed in the above knowledgebase article and restarting the computer the fourth member of the set magically appeared in /etc/certificates and I when I chose my GoDaddy cert in the Web Services site creator the port magically defaulted to the proper "443".
Everything working fine now!
Eureka!
