DNS not resolving

Question

Level 1

8 points

DNS not resolving

All of our OS X users in our workplace is not resolving local DNS settings of our servers after connecting to VPN. Other users include Linux and Windows users and it works for them smoothly.

Just a context: Our servers we're accessible via public domain before. Since we started securing everything, we proceed to put it behind firewall and only accessible via VPN. Since our developers are used to accessing the servers via domain name, we created a local DNS server for this. So when they connect to our VPN, DHCP + DNS settings would be passed to their machines automatically. Two DNS, one is for local resolv and one for public resolv. Unfortunately, all of our OS X users are experiencing problems. Linux and Windows machines aren't.

Server: riker.example.com -> 120.x.x.x (Public IP address)

-> 10.20.1.28 (Local IP address)

DNS settings that is passed to our users:

10.20.1.3 (for local)

8.8.8.8 (for public)

We have already checked:

DNS Server settings of our Fortigate device - we're 100% sure it is correct because our Linux and Windows users are working.
Firewall policies - again, 100% sure because it's working properly with Linux and Windows
Check /etc/resolv.conf files of the OS X machines - empty.
Flushed DNS cache using this: sudo killall -HUP mDNSResponder
Turned off firewall of OS X machines
Restarted mDNSresponder

After all those troubleshooting, it still does not work.

If i'm connected to VPN, it's still resolving the public ip address, and not the local ip address :

heineken:~ heineken$ ping riker.example.com

PING riker.example.com (120.x.x.x): 56 data bytes

Request timeout for icmp_seq 0

Request timeout for icmp_seq 1

^C

--- riker.example.come ping statistics ---

3 packets transmitted, 0 packets received, 100.0% packet loss

But if i ping local address:

heineken:~ heineken$ ping 10.20.1.28

PING 10.20.1.28 (10.20.1.28): 56 data bytes

64 bytes from 10.20.1.28: icmp_seq=0 ttl=63 time=62.714 ms

64 bytes from 10.20.1.28: icmp_seq=1 ttl=63 time=83.162 ms

^C

--- 10.20.1.28 ping statistics ---

2 packets transmitted, 2 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 62.714/72.938/83.162/10.224 ms

Checking by dig and nslookup:

heineken:~ heineken$ dig @10.20.1.3 riker.example.com

; <<>> DiG 9.8.3-P1 <<>> @10.20.1.3 riker.example.com

; (1 server found)

;; global options: +cmd

;; Got answer:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 64601

;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:

;riker.example.come. IN A

;; ANSWER SECTION:

riker.example.com. 3600 IN A 10.20.1.28

;; Query time: 58 msec

;; SERVER: 10.20.1.3#53(10.20.1.3)

;; WHEN: Fri Jun 10 11:33:00 2016

;; MSG SIZE rcvd: 52

heineken:~ heineken$ nslookup riker.example.com

Server: 10.20.1.3

Address: 10.20.1.3#53

Non-authoritative answer:

Name: riker.example.com

Address: 10.20.1.28

Contents of /etc/resolv.conf:

heineken:~ heineken$ cat /etc/resolv.conf

#

# Mac OS X Notice

#

# This file is not used by the host name and address resolution

# or the DNS query routing mechanisms used by most processes on

# this Mac OS X system.

#

# This file is automatically generated.

#

nameserver 10.20.1.3

nameserver 8.8.8.8

As you can see above, by pinging the domain it's still gives out the public IP address and not the local IP address.

But with dig and nslookup, it's already giving out the local IP Address.

Few more things I did:

Tried the solutions presented from here: http://apple.stackexchange.com/questions/26616/dns-not-resolving-on-mac-os-x - still did not work for us.
I installed a virtual machine - CentOS - inside an OS X machine and made it to have a bridge connection. It was resolving correctly the domain locally.
This means that Forticlient can be eliminated from the equation since the OS X and CentOS VM is just sharing the same adapter.
Seeked L3 support from Fortigate
- We did a lot of tcpdump on the Fortigate device to check traffic.
  - We discovered that when a Linux or a Windows machine pings a domain it seeks for DNS request from the Fortigate device first.
  - Unfortunately for the OS X machine, it does not do any DNS request from the Fortigate device. So what happens is that it still resolves to the public domain.
- I was advised by them that the issue is localized on the OS X devices because even after flushing cache, it does not perform any DNS query when pinging riker.example.com. So they cannot help us anymore.

This was tested on:

OS X versions: Yosemite 10.10.4, El Capitan 10.11.5

Fortigate 100D Firmware version: v5.2.4,build688 (GA)

Forticlient version: 5.4.0.493

Any advice?

Posted on Jun 9, 2016 9:17 PM

Reply

Answer 1

Slack84

Level 1

4 points

Jun 10, 2016 12:26 AM in response to kenanbiel

OS X is handling DNS differently compared to other OS.

The DNS list is not working as you might think, you entered two DNS Servers, your local DNS server 10.20.1.3 and Google DNS 8.8.8.8 for public, but this is not how OS X will deal with your DNS list entries.

Here is a short snipped from the OS X "resolver" documentation:

If there are multiple servers,

the resolver library queries them in the order listed. The algorithm used is to try a name server, and

if the query times out, try the next, until out of name servers, then repeat trying all the name

servers until a maximum number of retries are made.

This is still open to interpretation, the overall request might time out as the local DNS Server is not able to find external resources, but the query to the DNS server does not time out as the DNS server is responding, it is just unable to find the external resource.

You cannot set OS X to use one DNS for local queries and another for external queries as OSX tries to figure that out itself.

The correct setup in this case would be to only set the local DNS server in your clients to 10.20.1.3, and add a DNS forwarding in your local DNS server to an external DNS service like 8.8.8.8 from Google or your ISP's DNS server.

I also found another interesting thread regarding OS X and DNS handling, although this thread was in regards to 10.9, but I don't think is has changed much or at all.

http://apple.stackexchange.com/questions/113366/list-dns-servers-in-mavericks

Reply

Answer 2

kenanbiel Author

Level 1

8 points

Jun 10, 2016 1:49 AM in response to Slack84

Hi Slack,

I understand. I've mentioned that this only happens with OS X users. We have Linux and Windows users as well and they don't experience this issue. When those 3 different platforms connect to our VPN, the DNS settings is received by them just the same. Linux and Windows can resolve via ping riker.example.com getting the local IP 10.20.1.28.

The algorithm makes sense. Try the first one, if it fails then proceed to the next. That's why OS X is resolving with the public domain because it has failed DNS query (for some reason) with the first one then it proceed using Google DNS. My question is that, how come Linux and Windows users can resolve first via the local DNS server 10.20.1.3 but not with OS X?

Thanks!

Reply