Snow Leopard mini having DNS issues

I'm going to do 2 things I hate.

First is replying to my own post. It is usually a "duh" moment like now. I rebooted my system and the name resolution error went away. Why didn't I try that before posting in the first place.

The second is having a magic fix, like rebooting. Don't understand the underlying cause. Don't know why or how a reboot fixed it. Just magic happened. No configuration changes, no DNS changes, and everything now works correctly.

I can speculate that something was not quite fully configured on the first boot but was on the second. Then again monkeys could fly out my butt and I'd have just as many facts as to why that happened.

Unix Guru, I wouldn't be surprised if this problem started happening again. The system expects that all DNS servers configured in Network pref pane will return the same results, so it will sometimes query them in what seems to be a random order. The better fix would be to not have different DNS servers configured at the same time.

Just to add another data point. I have a Gentoo box running dnsmasq as my dhcp/dns server. This setup has been running fine for several years..

Immediately after I installed Snow Leopard on my 1-month old MBP, I was unable to ssh to one of my other Linux boxes.

I've made sure I had IPv6 turned off on both the MBP and the server. I have no other DNS servers configured. Dig resolves the dns name just fine. Its only when trying to connect with applications (ping, ssh, safari, etc) that the name resolution fails.

garion, can you post the output of the dig command?

Simple nuff:

vale:~ garion$ dig sx280-2

; <<>> DiG 9.6.0-APPLE-P2 <<>> sx280-2
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18720
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;sx280-2. IN A

;; ANSWER SECTION:
sx280-2. 86400 IN A 192.168.1.130

;; Query time: 5 msec
;; SERVER: 192.168.1.3#53(192.168.1.3)
;; WHEN: Mon Aug 31 22:54:26 2009
;; MSG SIZE rcvd: 41


And scutil --dns:

vale:~ garion$ scutil --dns
DNS configuration

resolver #1
domain : local
nameserver[0] : 192.168.1.3
order : 200000

resolver #2
domain : local
options : mdns
timeout : 2
order : 300000

resolver #3
domain : 254.169.in-addr.arpa
options : mdns
timeout : 2
order : 300200

resolver #4
domain : 8.e.f.ip6.arpa
options : mdns
timeout : 2
order : 300400

resolver #5
domain : 9.e.f.ip6.arpa
options : mdns
timeout : 2
order : 300600

resolver #6
domain : a.e.f.ip6.arpa
options : mdns
timeout : 2
order : 300800

resolver #7
domain : b.e.f.ip6.arpa
options : mdns
timeout : 2
order : 301000

OK, try this...

1. Run "sudo killall -USR1 mDNSResponder" to enable operation logging.
2. Run "sudo killall -USR2 mDNSResponder" to enable packet logging.
3. Run "sudo killall -HUP mDNSResponder" to clear the DNS cache.
4. Run "ping sx280-2" and wait for it to fail.
5. Post all the mDNSResponder logs in system.log from around the time of your test.

The logs will explain why it's failing.

Hi Snoop Dogg,

What is your take on the mDNSResponder logs I posted ?
Looks like a bug to me. The TTL value in a DNS response is 4 bytes.
What I noticed when browsing the sourcecode is that it is multiplied by 1000 to get the value in ticks instead of seconds. I am not 100% sure, but it looks like that multiplied value is stored again in a signed 32bits variable (which will be an issue for larger TTL values).

In the code for the logging, the received TTL value is maxed at 0x70000000 / 1000 (which explains the mismatch between the ttl in my zone file and the logged TTL value).

Should I file a defect for this ?

These are my logs covering an situation where I got a single ping and then lost name resolution again.

Firstly the pings (which include timestamps)

Tue 1 Sep 2009 21:36:26 EST
ping: cannot resolve mail.google.com: Unknown host
Tue 1 Sep 2009 21:36:36 EST
PING googlemail.l.google.com (66.249.89.18): 56 data bytes
64 bytes from 66.249.89.18: icmp_seq=0 ttl=245 time=131.184 ms

--- googlemail.l.google.com ping statistics ---
1 packets transmitted, 1 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 131.184/131.184/131.184/0.000 ms
Tue 1 Sep 2009 21:36:47 EST
ping: cannot resolve mail.google.com: Unknown host

Here are the mDNSResponder logs for the same time period:

Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Adding FD for uid 501
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceCreateConnection START
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000001
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr, 5000) START
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000001 (0)
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr) ADD 25 mail.google.com. CNAME googlemail.l.google.com.
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 0 googlemail.l.google.com. Addr
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Cancel 00000000 00000001
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) STOP
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000002
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr, 5000) START
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000002 (0)
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr) ADD 0 mail.google.com.home.gateway. Addr
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Cancel 00000000 00000002
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr) STOP
Sep 1 21:36:26 alloy mDNSResponder[18]: 78: Removing FD
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: Adding FD for uid 501
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceCreateConnection START
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000001
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr, 5000) START
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000001 (0)
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr) ADD 25 mail.google.com. CNAME googlemail.l.google.com.
Sep 1 21:36:36 alloy mDNSResponder[18]: -- Sent UDP DNS Query (flags 0100) RCODE: NoErr (0) RD ID: 40108 29 bytes from port 60782 to 10.1.1.1:53 --
Sep 1 21:36:36 alloy mDNSResponder[18]: 1 Questions
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 googlemail.l.google.com. Addr
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 Answers
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 Authorities
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 Additionals
Sep 1 21:36:36 alloy mDNSResponder[18]: --------------
Sep 1 21:36:36 alloy mDNSResponder[18]: -- Received UDP DNS Response (flags 8180) RCODE: NoErr (0) RD RA ID: 40108 93 bytes from 10.1.1.1:53 to 10.1.1.9:60782 --
Sep 1 21:36:36 alloy mDNSResponder[18]: 1 Questions
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 googlemail.l.google.com. Addr
Sep 1 21:36:36 alloy mDNSResponder[18]: 4 Answers
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 TTL1879048 4 googlemail.l.google.com. Addr 66.249.89.18
Sep 1 21:36:36 alloy mDNSResponder[18]: 1 TTL1879048 4 googlemail.l.google.com. Addr 66.249.89.83
Sep 1 21:36:36 alloy mDNSResponder[18]: 2 TTL1879048 4 googlemail.l.google.com. Addr 66.249.89.17
Sep 1 21:36:36 alloy mDNSResponder[18]: 3 TTL1879048 4 googlemail.l.google.com. Addr 66.249.89.19
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 Authorities
Sep 1 21:36:36 alloy mDNSResponder[18]: 0 Additionals
Sep 1 21:36:36 alloy mDNSResponder[18]: --------------
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 4 googlemail.l.google.com. Addr 66.249.89.18
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 4 googlemail.l.google.com. Addr 66.249.89.83
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 4 googlemail.l.google.com. Addr 66.249.89.17
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 4 googlemail.l.google.com. Addr 66.249.89.19
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: Cancel 00000000 00000001
Sep 1 21:36:36 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) STOP
Sep 1 21:36:37 alloy mDNSResponder[18]: 78: Removing FD
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Adding FD for uid 501
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceCreateConnection START
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000001
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr, 5000) START
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000001 (0)
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr) ADD 25 mail.google.com. CNAME googlemail.l.google.com.
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 0 googlemail.l.google.com. Addr
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Cancel 00000000 00000001
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) STOP
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000002
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr, 5000) START
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000002 (0)
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr) ADD 0 mail.google.com.home.gateway. Addr
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Cancel 00000000 00000002
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr) STOP
Sep 1 21:36:47 alloy mDNSResponder[18]: 78: Removing FD
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Adding FD for uid 501
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceCreateConnection START
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000001
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr, 5000) START
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000001 (0)
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com., Addr) ADD 25 mail.google.com. CNAME googlemail.l.google.com.
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) ADD 0 googlemail.l.google.com. Addr
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Cancel 00000000 00000001
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(googlemail.l.google.com., Addr) STOP
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Error socket 81 created 00000000 00000002
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr, 5000) START
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Error socket 81 closed 00000000 00000002 (0)
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr) ADD 0 mail.google.com.home.gateway. Addr
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Cancel 00000000 00000002
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: DNSServiceQueryRecord(mail.google.com.home.gateway., Addr) STOP
Sep 1 21:36:57 alloy mDNSResponder[18]: 78: Removing FD

Any ideas?

BTW, I'm pretty sure this problem also affects my iPhone on my home network.

Hi Coldboot,

Looks like you have the same high/max-out TTL values (0x70000000 / 1000 = 1879048) in your logs which cause 10.6 to add a negative entry in its cache:

Sep 1 21:36:36 alloy mDNSResponder18: 0 TTL1879048 4 googlemail.l.google.com. Addr 66.249.89.18

If you can infulence/change this in your DNS server I am pretty sure this will fix your problem
(however this might not be possible for a built-in forwarding DNS server in a dsl-router).

I've updated my modem firmware and that seems to have fixed the issue.

The firmware release notes contain this tidbit:

Change the DNSMasq version from v2.43 to v2.40 for iPhone issue.

So perhaps that affects DNS TTL values?

I'll keep an eye on it - thanks for the help.

Ok, I've posted my system.log here: http://pastebin.ca/1550324

It looks like its searching a dns server via bonjour type records (lb. dns-sd.udp) for some reason, never searching for the requested info.

I do have avahi also running on my gentoo box, advertising my printer and a few other things. (as seen in the long).

What sort of TTL value do you recommend changing to?

Well, I changed mine from 2592000 (=30days, which is admittedly quite high 🙂 ) to 86400 (=1day), which solved my problem.

I also found some lines in the mDNSResponder source which seem to top any values higher than 1879048 seconds (= 0x70000000 / 0d1000)

Hi garion911,

you seem to have a different issue. Looks like your domain is .local so the resolver is trying to get the SOA record for the .local zone:

Sent UDP DNS Query (flags 0100) RCODE: NoErr (0) RD ID: 63241 11 bytes from port 54948 to 192.168.1.3:53 --
Sep 1 07:22:26 vale mDNSResponder[26]: 1 Questions
Sep 1 07:22:26 vale mDNSResponder[26]: 0 local. SOA

Your DNS server apparently does not have this zone so responds with a NXDomain

Received UDP DNS Response (flags 8183) RCODE: NXDomain (3) RD RA ID: 63241 11 bytes from 192.168.1.3:53 to 192.168.1.88:54948 --

You should probably change the "search domains" setting in your clients network configuration.

Does the problem also occur with small TTL values? I think ours are set to 300 seconds.