Push Forward Proxy info within DHCP

Hello Kostas

I think John Lockwood amongst others knows this better than most. Search for thread using his name. It's possible this has already been discussed? If not he may spot this thread (hopefully) and offer more assistance? He's also been extremely generous to make available the utility he uses:

http://web.me.com/jelockwood/MyUtilities/dhcp.html

HTH?

Tony

Kostas B wrote:
Hello,

Is it possible to "push" Forward proxy info to clients using DHCP (with some option)?

Regards

Kostas

It is possible to use my DHCP Option Code Utility to create the required DHCP option codes for advertising a proxy server. See http://en.wikipedia.org/wiki/WebProxy_AutodiscoveryProtocol

As per the above article, there are two methods for a client to 'discover' a proxy server, one is via a DHCP option code, as per your query, the other is via a standard URL formed based on the clients domain name, e.g. http://wpad.example.com/wpad.dat

Unfortunately, it is my impression that while it is possible to set a Mac OS X Server to provide both the DHCP option code, and to host a suitable file at the address specified via the DNS method, I don't believe any version of Mac OS X as a client will use either of these methods.

You can manually configure a Mac OS X client to look for the DNS address and then the wpad file in System Preferences -> Network -> Advanced… -> Proxies -> Automatic Proxy Configuration , you could in theory use Managed Preferences via Workgroup Manager as well. However both these methods cause problems for laptops since when out of the office they are unlikely to be able to access your proxy server, or want to.

So, these two methods on a Mac OS X Server may work with Windows clients (if Windows still supports these methods), but as far as I am aware are useless for Mac OS X clients.

For the benefit of others, a typical reason for using a proxy server, is to monitor network traffic especially web-browsing. An alternate approach that should work for Macs, would be to have a device in-between the LAN and the Internet so all web-browsing has to go through it regardless. This should therefore not need the Macs to be specifically configured.

As we all know, historically Apple has not addressed the needs of the Enterprise market and this could be considered an example of that.

For the benefit of everyone, here is an example DHCP option code as you would add it to your bootpd.plist file at /etc/bootpd.plist

<code><key>dhcpoption252</key>
<data>
aHR0cDovL3dwYWQuZXhhbXBsZS5jb20vd3BhZC5kYXQ=
</data>

The above uses the example, URL of http://wpad.example.com/wpad.dat obviously you need to use my utility to create a value that matches your domain and to have a web-server on your LAN that can host that file. The file then tells the client how to access the proxy server, again see the above wiki article and also this one http://en.wikipedia.org/wiki/Proxy_auto-config which tells you how to write the wpad.dat (aka. proxy.pac) file.

Update: I notice Mac OS X 10.6 now has a "Auto Proxy Discover" option, as Apple do not provide any real documentation for this level of detail, only they know for sure what this does. However it does sound like it might implement the DHCP and DNS methods of auto-discovery. This option was not present in Mac OS X 10.5. If this is what it does you do not need to turn on and fill in the "Automatic Proxy Configuration" option.

John Lockwood wrote:
Update: I notice Mac OS X 10.6 now has a "Auto Proxy Discover" option, as Apple do not provide any real documentation for this level of detail, only they know for sure what this does. However it does sound like it might implement the DHCP and DNS methods of auto-discovery. This option was not present in Mac OS X 10.5. If this is what it does you do not need to turn on and fill in the "Automatic Proxy Configuration" option.

If Apple have added support for WPAD with this option then I am doing pretty good in getting Apple to adopt my Mac OS X Server suggestions 🙂 they previously added support for DHCP option codes - even if they did not provide a means for creating them, and also previously changed the default configuration for Squirrelmail based on my suggested settings so it matched the IMAP folder names Apple Mail uses as default. So adding WPAD is another one ticked off on the list.

Hi John

Would just like to add a bit of info I found through a lengthy troubleshooting process. Maybe you can feed that back into Apple also 🙂

(Technical explanation to follow, summary towards the end)

I suspect part of the reason many people can't get the WPAD DHCP mechanism to work, is because they're using Windows-based DHCP servers - not an uncommon thing in many corporate environments.

The problem is seemingly due to an incompatibility between Microsoft DHCP and OSX' WPAD implementation. (BTW, OS X does implement WPAD properly, contrary to some claims by the majority of Google posts on the topic.)

The WPAD protocol first tries settings supplied in DHCP - option 252 - and then falls back to trying to resolve through DNS if that fails (i.e. the default of http://wpad/wpad.dat.)

Trouble is, when Microsoft's DHCP implementation (and possibly other implementations that interpret the RFC the same) supply a string in a DHCP option, it null-terminates it. The DHCP RFC says that this "should" not be done (i.e. it's permitted but not recommended.) However, the RFC also says that clients that receive options "should" cater for the possibility of the string being null-terminated.

So what it seems is that neither Windows Server nor OS X play ball - Windows passes OS X a null terminated string and OSX interprets it literally, appending a double-escaped null character to the URL (%25 in a URL normalises to the "%" character, and %00 is the null character. So %2500 = %%00 which gets interpreted as null. This is usually visible inside the web server logs from where OS X attempts to download the WPAD.DAT file.

--------------------Summary/workaround--------------------
So, to cut a very long story short, the workaround is to remove Option 252 from the DHCP server and rely solely on DNS resolution - i.e. the client must be able to resolve the "wpad" shortname. Also, unlike Windows clients, OS X seems to cache the option from when it receives its IP address (rather than retrieving it again when the browser launches), so to get rid of the option you will need to do something like disconnecting/reconnecting the cable to refresh the DHCP options.

This will allow OS X - as a client - to use WPAD without the need to specify any special settings other than "Autodetect proxy." This achieves the goal of having a common proxy setting for both office and home Internet access.

JannieH wrote:
Hi John

Would just like to add a bit of info I found through a lengthy troubleshooting process. Maybe you can feed that back into Apple also 🙂

(Technical explanation to follow, summary towards the end)

I suspect part of the reason many people can't get the WPAD DHCP mechanism to work, is because they're using Windows-based DHCP servers - not an uncommon thing in many corporate environments.

The problem is seemingly due to an incompatibility between Microsoft DHCP and OSX' WPAD implementation. (BTW, OS X does implement WPAD properly, contrary to some claims by the majority of Google posts on the topic.)

The WPAD protocol first tries settings supplied in DHCP - option 252 - and then falls back to trying to resolve through DNS if that fails (i.e. the default of http://wpad/wpad.dat.)

Trouble is, when Microsoft's DHCP implementation (and possibly other implementations that interpret the RFC the same) supply a string in a DHCP option, it null-terminates it. The DHCP RFC says that this "should" not be done (i.e. it's permitted but not recommended.) However, the RFC also says that clients that receive options "should" cater for the possibility of the string being null-terminated.

Thanks for the info. If I understand correctly it sounds like the fault is Apple's DHCP client not ignoring the null termination character and therefore a 'proper' solution would be for Apple to modify their DHCP client to strip off the null character.

Would a workaround of setting up a HTML redirect of the (HTML escaped) URL including the null terminator work? That is redirect

http://wpad/wpad.dat%2500 to http://wpad/wpad.dat

For those interested, if you added a DHCP option code to Apple's DHCP server of type 'string' it would normally not be null terminated, this includes if you used my DHCP Option Code Utility to generate it. If however you entered it as Hex using my utility, you could include the null termination if you desired.

Finally, are you able to confirm that Mac OS X 10.6 correctly supports the DNS method of finding a proxy server?

Apologies for the delay in responding.

Correct, it does seem that the only realistic way to resolve this permanently would be for Apple to modify DHCP client behaviour.

Also, I can confirm that DNS resolution of WPAD works properly provided that DHCP option 252 is not set, as OSX correctly gives DHCP discovery priority.

Unfortunately I don't know the OSX version of the machine we tested on - I'm a Windows guy that was just doing troubleshooting, so didn't think to check! We did the testing in early October 2010, so presumably whatever was current at the time.

An HTTP redirect might work - I tried it, but the Windows IIS web server I was using didn't take too kindly to null characters, so wasn't able to use that option as a workaround myself. But it should be possible.

JannieH wrote:
Apologies for the delay in responding.

Correct, it does seem that the only realistic way to resolve this permanently would be for Apple to modify DHCP client behaviour.

Also, I can confirm that DNS resolution of WPAD works properly provided that DHCP option 252 is not set, as OSX correctly gives DHCP discovery priority.

Unfortunately I don't know the OSX version of the machine we tested on - I'm a Windows guy that was just doing troubleshooting, so didn't think to check! We did the testing in early October 2010, so presumably whatever was current at the time.

An HTTP redirect might work - I tried it, but the Windows IIS web server I was using didn't take too kindly to null characters, so wasn't able to use that option as a workaround myself. But it should be possible.

If you have access to do further testing, you could try using my DHCP Utility to add a hex encoded value equivalent to the desired string with the hex 00 (a NULL character) added. If the Windows clients need the null termination this may satisfy them, and your testing will establish whether Macs are happy as well.

The following site is great for converting ASCII to Hex, http://www.easycalculation.com/ascii-hex.php remember to add a hex 00 to the end to represent a NULL character and then enter the whole lot as Hex in my utility to generate a value to use with Apple's DHCP server.

My DHCP Option Code Utility (hopefully) can still be downloaded via http://web.me.com/jelockwood/MyUtilities/dhcp.html

I'm a mere Windows admin so don't have access to an OS X server (also I suspect the problem doesn't exist if using an OS X client with an OS X server.)

However, I have recently had opportunity to mess about with a Linux-based DHCP server on my home router and can confirm that Windows is quite happy if the DHCP Option 252 string is returned without null termination; i.e. it doesn't care if the string is null terminated or not, whereas OS X (as client) currently insists that the string must NOT be null-terminated.

Thanks to all of you for this informative post. I've just configured squid as a web cache proxy (no authentication) on my Xserve running OS X 10.6.5 and I'd love to be able to force all DHCP users (which are all connected exclusively wirelessly) to use the proxy, but it sounds like this can't be done via DHCP options since I can't mandate that they all enable "Auto Proxy Discovery". We have lots of occasions where guests come and need wireless access and establishing requirements for custom configuration of network settings would be met with contempt. I'd appreciate it if you could confirm that my understanding is correct about this not being able to be handled without client configuration. I'm serving DHCP from the Xserve for the subnet in question and the Xserve's also the OD Master with Radius enabled for some of our Netgear WG102 WAPs. I would love to take advantage of the web cache proxy given that our bandwidth is limited. Anyone know if there's a way to create a default "guest" account in Workgroup Manager that could be invoked when someone connects without authentication via DHCP on this subnet? Another way to phrase my question is to ask if there's a way that all of my DHCP users could transparently be "authenticated" automatically as that specific user so that settings could be established pointing that user to our proxy by default? I know this might sound strange, but I'm try to think if there might be any other way to force DHCP users to use our proxy without requiring custom network settings on the client side. If there is a way to handle this, the next question would be whether it would work regardless of client operating system; since our guests could be using any operating system. Thanks again, and please let me know if you need any further clarification on my setup or my question.