DNS / Reverse DNS name mismatch after certificate update

Question

Level 1

101 points

DNS / Reverse DNS name mismatch after certificate update

I'm running 10.9.5 with Server 3.2.2, using a SSL certificate from a third-party, with a true IP address with DNS properly set up for this.

A couple of weeks ago, the certificate was going to expire, so I renewed it (using the web interface of my hosting company) and added the new certificate (and intermediate) to OS X server, and deleted the old one, then rebooted.

Since that change, every once in a while, we are getting warning messages about a server name mismatch: the error message indicates that the server name doesn't match the DNS name, and shows the correct certificate, along with the REVERSE DNS name of my IP address. The IP address is a Time Warner business account and thus the RDNS doesn't match the DNS.

Basically, it says something like this "The server a.b.c.d.timewaernerbusiness.com may be pretending to be myDomain.com" where a.b.c.d is my actual IP address.

I could tell my users "just trust the mismatch" but that's not good policy : I'd like to fix it.

Additional complexities:

* This is the 3rd time I've updated the certificate. The first time it went smoothly with none of these problems.

* This new certificate is from a different registrar (COMODO instead of Verisign)

* The error only crops up occasionally, and it seems to only happen on the Calendar service.

Any ideas?

I could probably solve this by having Time Warner set up RDNS properly for my IP address - but I never had to do this in the 2+ years prior, so I'm not sure that's the right solution.

Mac mini, OS X Server, OSX 10.9.4 with Server 3.2

Posted on Mar 5, 2015 5:02 PM

Reply

Answer 1

Mar 6, 2015 5:15 AM in response to xmddmx

If you run your own mail server then having the reverse DNS record correct is actually quite important as this is used as part of determining the trustworthiness of emails coming from your server i.e. how spammy they are.

I have just rechecked mine and fortunately the reverse DNS is correct - even as far as the rest of the world is concerned.

Note: If you run your own internal DNS server then you may also have your own internal reverse DNS records. To properly test how the rest of the world see things try something like this in Terminal.app

nslookup mypublicip 8.8.8.8

Where 8.8.8.8 is one of Google's DNS servers.

Reply

Answer 2

xmddmx Author

Level 1

101 points

Mar 6, 2015 7:00 AM in response to John Lockwood

I agree about the importance of reverse dns matching forward dns, but I'm not running a mail server at this time.

The reverse DNS is accurate (both inside the LAN and from outside on the WAN) - that's not the issue.

The issue is that for some reason certain services (Calendar) on clients seem to be checking the certificate against the RDNS name, rather than the DNS name. This seemed to happen at the same time that I renewed my certificate, so that seems the most likely cause, but I don't know that for sure.

Reply

Answer 3

xmddmx Author

Level 1

101 points

Mar 6, 2015 7:15 AM in response to xmddmx

More info:

Although OS X Server.app is configured to use the new certificate (from COMODO) I still have the old one (from PositiveSSL CA) in my keychain.

The old one says "This certificate is marked as trusted for 127.0.0.1". The new one does not.

Also, Console.log shows hundreds of messages like these:

3/6/15 7:12:55.423 AM secd[497]: SecErrorGetOSStatus unknown error domain: com.apple.security.sos.error for error: The operation couldn’t be completed. (com.apple.security.sos.error error 2 - Public Key not available - failed to register before call)

3/6/15 7:12:55.423 AM secd[497]: securityd_xpc_dictionary_handler Keychain Access[441] DeviceInCircle The operation couldn’t be completed. (com.apple.security.sos.error error 2 - Public Key not available - failed to register before call)

Reply

Answer 4

xmddmx Author

Level 1

101 points

Mar 6, 2015 8:33 AM in response to xmddmx

I tried a different approach and may have solved it.

My Hosting company provides certificates, so the process of generating a certificate request is handled by them, and they simply send me the certificates.

When I renewed the certificate I did these steps:

What does NOT work:

Server App / Certificates / click the + button and choose "Import a Certificate Identity..."
Drag the 3 files (private key, certificate, intermediate) to the window
Settings: change "Secure services using" to my new certificate.

This process seems to have confused OS X server, perhaps because the new certificate is the same name as the old one?

In any case, a different set of steps seems to have fixed the issue:

What DOES Work:

Go to Server App / Certificates
Double-click the certificate which is about to expire
Scroll to the bottom and click the "Renew" button.
Create the CSR as per the instructions (since I already have the renewed certificate, I just entered dummy values here)
This will then give you a new "Pending" certificate in the Certificates list.
Open the Pending certificate, and drag & drop your 3 certificate items (private, public key, and intermediates)

Reply

Answer 5

xmddmx Author

Level 1

101 points

Mar 10, 2015 7:26 PM in response to xmddmx

Update: This problem seems to be intermittent, and unfortunately my solution doesn't seem to work.

I went back a second time and forced the DNS server to only have the external (WAN) IP address, and made sure that it also has a reverse DNS entry.

Results: still fails.

So I'm kind of stumped.

This worked fine for a year on 10.9.x.

It broke around the time two things happened:

1. I got a new cable modem from my ISP (but the static IP address remained the same)

2. I upgraded the SSL certificate (from Comodo instead of Verisign).

(p.s. In these forums, is there any way to remove the "this solved my question" marker?)

Reply