realzcubed

Q: Privacy Enhancing Filtering Proxy Chain for OS X

A privacy enhanced web proxy is a nearly essential tool on the modern web: it blocks ads, malicious scripts, and conceals information used to track you around the web. I've provided a quick setup below in case it's useful to others. This will build a privatizing squid:privoxy proxy chain that works with any browser, and can be used by anyone on your LAN, including and especially secure VPN logins and ssh tunnels. In my experience, this setup is a lot more capable and effective than using a simple adblocking Firefox Add-On. There's a world of difference between reading ad-filled web pages with and without a filtering proxy server. I've also included information for a polipo proxy that can be used with Tor for full anonymity, as well as a script for ssh tunnelling

 

  • Install Xcode and Macports
  • Install squid, privoxy, and polipo:

 

$ sudo port selfupdate

$ sudo port install squid privoxy polipo

$ sudo port load squid privoxy polipo

 

  • Configure the squid/privoxy/polipo config files shown below, then relaunch the proxies and test to make sure they're up:

 

$ sudo launchctl unload -w /Library/LaunchDaemons/org.macports.Squid.plist

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Squid.plist

$ sudo launchctl unload -w /Library/LaunchDaemons/org.macports.Privoxy.plist

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Privoxy.plist

$ sudo launchctl unload -w /Library/LaunchDaemons/org.macports.Polipo.plist

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Polipo.plist

$ nmap -p 3128,8118,8123 localhost

Starting Nmap 5.51 ( http://nmap.org ) at 2012-02-07 11:47 EST

Nmap scan report for localhost (127.0.0.1)

Host is up (0.00013s latency).

PORT     STATE SERVICE

3128/tcp open  squid-http

8118/tcp open  privoxy

8123/tcp open  polipo

 

  • Now web applications can use your filtering web proxy chain. If you use the config files below, websites will not know where you came from (HTTP_REFERER header is forged), and will not know your User Agent (also forged), and read access is block to several HTTP header fields. Ads are filtered. Your connection looks like this:

 

Application  <--port 3128-->  Squid  <--port 8118--> Privoxy  <----> Internet

 

  • Configure your network to add an option to route your web traffic through this proxy. System Preferences>Network>Wi-Fi/Ethernet/...>Locations:>Edit Locations...> Gear icon, Duplicate Location, Advanced...>Proxies> Check boxes for HTTP and HTTPS web proxies with proxy server localhost:3128.
  • While you're at it, configure your OS and browsers to block Adobe flash cookies. Read this WSJ article series to understand how this impacts your privacy.
    • System Preferences>Flash Player>Block all sites from storing information, using your camera and microphone, and networking with peers. Also Delete all data and go to this Adobe Flash Player Settings web page and block all sites from storing information, using your camera and microphone, and networking with peers.
    • Firefox/Safari>DO NOT ALLOW third party cookies, request not to be tracked
    • Firefox Add-Ons: NoScript (blocks/manages JavaScript), Beef TACO (blocks/manages flash cookies), BetterPrivacy (blocks/manages flash cookies), and the EFFs HTTPS Everywhere.
    • You can also download the Tor anonymous proxy chain for both OS X and iOS devices. This will run a little polipo proxy natively on mobile devices.

 

Here are the config file settings. Search through the config file too see the appropriate location for these settings. Turn off http_access and icp_access (squid), permit-access (privoxy), and allowedClients (polipo) if you do not want everyone on your LAN to be able to use the proxy. Double check that you're not running an open web proxy on the internet.

 

$ sudo vi /opt/local/etc/squid/squid.conf

 

# See http://www.privoxy.org/user-manual/config.html

# Define Privoxy as parent proxy (without ICP)

cache_peer 127.0.0.1 parent 8118 7 no-query

 

http_access allow localnet

icp_access allow localnet

 

via off

 

# old 'http_anonymizer standard'

header_access From deny all

# forge Referer in Privoxy

# header_access Referer deny all

header_access Server deny all

# forge User-Agent in Privoxy

# header_access User-Agent deny all

header_access WWW-Authenticate deny all

header_access Link deny all

 

# more privacy

header_access Cache-Control deny all

header_access Proxy-Connection deny all

header_access X-Cache deny all

header_access X-Cache-Lookup deny all

header_access Via deny all

header_access Forwarded-For deny all

header_access X-Forwarded-For deny all

header_access Pragma deny all

header_access Keep-Alive deny all

 

shutdown_lifetime 10 seconds

 

# See http://www.privoxy.org/user-manual/config.html

# Define ACL for protocol FTP

acl ftp proto FTP

 

# Do not forward FTP requests to Privoxy

always_direct allow ftp

 

# See http://www.privoxy.org/user-manual/config.html

# Forward all the rest to Privoxy

never_direct allow all

 

dns_nameservers 10.0.1.2 10.0.1.1

forwarded_for off

 

$ sudo vi /opt/local/etc/privoxy/config

 

forward  /      .

 

$ sudo vi /opt/local/etc/privoxy/match-all.action

 

{ \

+change-x-forwarded-for{block} \

+deanimate-gifs{last} \

+filter{refresh-tags} \

+filter{img-reorder} \

+filter{banners-by-size} \

+filter{webbugs} \

+filter{jumping-windows} \

+filter{ie-exploits} \

+hide-from-header{block} \

+hide-referrer{conditional-block} \

+session-cookies-only \

+set-image-blocker{pattern} \

}

/ # Match all URLs

# See http://www.christianschenk.org/blog/enhancing-your-privacy-using-squid-and-privo xy/

{ \

+hide-referrer{conditional-forge} \

+hide-user-agent{Mozilla/5.0} \

}

/ # Match all URLs

 

$ sudo vi /opt/local/etc/privoxy/user.action

 

# fix bing's travel site, others

{ -block }

ads1.msn.com/

.bing.com/travel/jsxc\.vjs\?

.onecause.com

.apple.com

.go.com

 

# sourceforge

{ -block -filter -deanimate-gifs}

.sourceforge.net

.dell.com

 

# expedia

{ -hide-user-agent }

.expedia.com

 

# don't filter downloads

{-filter -deanimate-gifs}

/.*\.iso(\?|$)

/.*\.mp3(\?|$)

/.*\.mp4(\?|$)

/.*\.mov(\?|$)

/.*\.mpg(\?|$)

/.*\.ogg(\?|$)

/.*\.aac(\?|$)

/.*\.zip(\?|$)

/.*\.pdf(\?|$)

/.*\.dmg(\?|$)

/.*\.tar(\?|$)

/.*\.gz(\?|$)

/.*\.dat(\?|$)

 

$ sudo vi /opt/local/etc/privoxy/config

 

proxyAddress = "0.0.0.0"    # IPv4 only

allowedClients = 127.0.0.1, 10.0.1.0/16

Mac mini, Mac OS X (10.7.3), Lion Server, 8 GB

Posted on Feb 7, 2012 9:46 AM

Close

Q: Privacy Enhancing Filtering Proxy Chain for OS X

  • All replies
  • Helpful answers

  • by tangucho,

    tangucho tangucho Mar 27, 2012 11:22 AM in response to realzcubed
    Level 1 (0 points)
    Mar 27, 2012 11:22 AM in response to realzcubed

    Awesome post! Thank you a lot!

     

    I do beleive that if everyone should be concerned about privacy on the internet.

     

    Can you give some guidelines about the same configuration but using:

     

    Squidman (Squid GUI)

    Tor Firefox browser bundle

    I2P network

    Freenet.

     

     

    Maybe just using privoxy to forward ports properly? I am a beginner so any help would be great.

     

    Thanks in advance!

  • by realzcubed,

    realzcubed realzcubed May 27, 2012 5:35 PM in response to realzcubed
    Level 1 (1 points)
    May 27, 2012 5:35 PM in response to realzcubed

    Two comments:

     

    1. Here's a proxy.pac file to use with iOS/other on your LAN. Bypasses the proxy for all LAN traffic, uses proxy for all other. Put this up on your local webserver, assuming dns resolves server.com to your server's LAN IP.

     

    function FindProxyForURL(url, host)

    {

    if (

       isPlainHostName(host) ||

       shExpMatch(host, "192.168.*") ||

       shExpMatch(host, "172.16.*") ||

       shExpMatch(host, "127.*") ||

       shExpMatch(host, "10.*") ||

       dnsDomainIs(host, ".server.com") ||

       dnsDomainIs(host, ".LOCAL") ||

       dnsDomainIs(host, ".local")  &amp;&amp;

            !dnsDomainIs(host, ".server.com") ||

       (url.substring(0,3) == "ftp")

    )

            return "DIRECT";

    else

            return "PROXY server.com:3128";

    }

     

    2. Comment out "head_access WWW-Authenticate deny all" to avoid breaking .htaccess web logins.

  • by realzcubed,

    realzcubed realzcubed May 27, 2012 5:41 PM in response to realzcubed
    Level 1 (1 points)
    May 27, 2012 5:41 PM in response to realzcubed

    And this Privoxy setting will make all clients look like iPads, and probably has less Panopticlick entropy [https://panopticlick.eff.org/] than the generic User Agent given above:

     

    +hide-user-agent{Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5}

  • by Franck P,

    Franck P Franck P Aug 23, 2012 8:00 AM in response to realzcubed
    Level 1 (0 points)
    Aug 23, 2012 8:00 AM in response to realzcubed

    This configuration looks great and I was try to apply for my laptop. Unfortunatly I'm not an expert, and I have problem with config file settings for squid.config.

    I was installing squid (at first 2.7 version but later 3.1, because being able to use the GUI squidMan)), Privoxy and polipo with sucess with MacPorts. Using also MacPort to get nmap.and proxies look to be up :

     

    Starting Nmap 6.01 ( http://nmap.org ) at 2012-08-23 21:59 PHT

    Nmap scan report for localhost (127.0.0.1)

    Host is up (0.00046s latency).

    PORT     STATE SERVICE

    3128/tcp open  squid-http

    8118/tcp open  privoxy

    8123/tcp open  polipo

     

    Configure the network was not a problem (just an interrogation about FTP proxy ?)

     

    To edit and add lines and save  match-all.action,user.action

    was fine also. I don't know why the command sudo vi /opt/local/etc/privoxy/config is repeat twice one to add forward  / and later 

    proxyAddress = "0.0.0.0"    # IPv4 only

    allowedClients = 127.0.0.1, 10.0.1.0/16

    I was add these 3 lines anyway, the main problem being I guess to put properly configurations  for squid.conf

     

    Here below the template gave by SquidMan,(easier for me getting the main lines!) I just have modified Privoxy as parent proxy but I was not able to manage properly where adding these settings.( getting error about localhost ie).

    Could you kindly past them in this template ? I guess it will fix my configuration ! thank you in advance.

    Sincerly,

    Franck

     

    # ----------------------------------------------------------------------

    # WARNING - do not edit this template unless you know what you are doing

    # ----------------------------------------------------------------------

     

     

    # the parent cache

    cache_peer 127.0.0.1 parent 8118 7 no-query no-digest no-netdb-exchange default

     

     

     

     

    # disk and memory cache settings

    cache_dir ufs %CACHEDIR% %CACHESIZE% 16 256

    maximum_object_size %MAXOBJECTSIZE%

     

     

     

     

    # store coredumps in the first cache dir

    coredump_dir %CACHEDIR%

     

     

     

     

    # the hostname squid displays in error messages

    visible_hostname %VISIBLEHOSTNAME%

     

     

     

     

    # log & process ID file details

    cache_access_log %ACCESSLOG%

    cache_log %CACHELOG%

    cache_store_log %STORELOG%

    pid_filename %PIDFILE%

     

     

     

     

    # Squid listening port

    http_port %PORT%

     

     

     

     

    # Access Control lists

    acl localhost src 127.0.0.1/32

    acl to_localhost dst 127.0.0.0/8 0.0.0.0/32

    acl manager proto cache_object

    acl SSL_ports port 443

    acl Safe_ports port 80                    # http

    acl Safe_ports port 21                    # ftp

    acl Safe_ports port 443                    # https

    acl Safe_ports port 70                    # gopher

    acl Safe_ports port 210                    # wais

    acl Safe_ports port 1025-65535          # unregistered ports

    acl Safe_ports port 280                    # http-mgmt

    acl Safe_ports port 488                    # gss-http

    acl Safe_ports port 591                    # filemaker

    acl Safe_ports port 777                    # multiling http

    acl CONNECT method CONNECT

    %ALLOWEDHOSTS%

    %DIRECTHOSTS%

     

     

     

     

    # Only allow cachemgr access from localhost

    http_access allow manager localhost

    http_access deny manager

     

     

     

     

    # Deny requests to certain unsafe ports

    http_access deny !Safe_ports

     

     

     

     

    # Deny CONNECT to other than secure SSL ports

    http_access deny CONNECT !SSL_ports

     

     

     

     

    # protect web apps running on the proxy host from external users

    http_access deny to_localhost

     

     

     

     

    # rules for client access go here

    http_access allow localhost

    %HTTPACCESSALLOWED%

     

     

     

     

    # after allowed hosts, deny all other access to this proxy

    # don't list any other access settings below this point

    http_access deny all

     

     

     

     

    # specify which hosts have direct access (bypassing the parent proxy)

    %ALWAYSDIRECT%

    always_direct deny all

     

     

     

     

    # hierarchy stop list (squid-recommended)

    hierarchy_stoplist cgi-bin ?

     

     

     

     

     

     

    # refresh patterns (squid-recommended)

    refresh_pattern ^ftp:                    1440          20%          10080

    refresh_pattern ^gopher:          1440          0%          1440

    refresh_pattern -i (/cgi-bin/|\?) 0          0%          0

    refresh_pattern .                    0          20%          4320

  • by Cachibache,

    Cachibache Cachibache Sep 25, 2012 9:44 PM in response to realzcubed
    Level 1 (0 points)
    Sep 25, 2012 9:44 PM in response to realzcubed

    Hello,

     

    This is way over my head and got to your post by searching "X-Forwarded-For" which I saw being used on a FireFox browser to conceal the geographical location when login in a website outside the US. I am looking for that same capability using Safari.

     

    Will this script accomplish this among onther things? If so can you choose which IP address you "appear as"?

     

    Thank you

  • by essandess,

    essandess essandess Feb 11, 2013 2:27 PM in response to Cachibache
    Level 1 (28 points)
    Applications
    Feb 11, 2013 2:27 PM in response to Cachibache

    can you choose which IP address you "appear as"?

     

    No -- you must use tor or a VPN tunnel for that. If the remote server doesn't know the ip you're attached to, its packets will never be routed back to you.

  • by essandess,

    essandess essandess Feb 11, 2013 2:37 PM in response to realzcubed
    Level 1 (28 points)
    Applications
    Feb 11, 2013 2:37 PM in response to realzcubed

    This setup breaks a recent configuration change in Photo Stream. I had to break out wireshark to figure it out. The following addition to Privoxy's user.action file fixes Photo Stream (as well as automated TV Guide listings downloaded by EyeTV). I didn't attempt to minimize the block removals, but this works.

     

    $ sudo vi /opt/local/etc/privoxy/user.action


    # fix icloud, photostream

    { -block -filter -hide-user-agent -deanimate-gifs -hide-from-header -set-image-blocker }

    TAG:^User-Agent: PhotoStreamAgent/

    .apple.com

    .icloud.com

    .amazonaws.com

     

    # TV Guide listings on EyeTV

    { -hide-user-agent }

    .macrovision.com

     

    Also, use this User-Agent for an iOS 6.1 passport:

     

    $ sudo vi /opt/local/etc/privoxy/match-all.action

     

    { \

    +hide-referrer{conditional-forge} \

    +hide-user-agent{Mozilla/5.0 (iPad; CPU OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B141 Safari/8536.25} \

    }

    / # Match all URLs