Privacy Enhancing Filtering Proxy Chain for OS X

Question

Level 1

2 points

Privacy Enhancing Filtering Proxy Chain for OS X

A privacy enhanced web proxy is a nearly essential tool on the modern web: it blocks ads, malicious scripts, and conceals information used to track you around the web. I've provided a quick setup below in case it's useful to others. This will build a privatizing squid:privoxy proxy chain that works with any browser, and can be used by anyone on your LAN, including and especially secure VPN logins and ssh tunnels. In my experience, this setup is a lot more capable and effective than using a simple adblocking Firefox Add-On. There's a world of difference between reading ad-filled web pages with and without a filtering proxy server. I've also included information for a polipo proxy that can be used with Tor for full anonymity, as well as a script for ssh tunnelling

Install Xcode and Macports
Install squid, privoxy, and polipo:

$ sudo port selfupdate

$ sudo port install squid privoxy polipo

$ sudo port load squid privoxy polipo

Configure the squid/privoxy/polipo config files shown below, then relaunch the proxies and test to make sure they're up:

$ sudo launchctl unload -w /Library/LaunchDaemons/org.macports.Squid.plist

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Squid.plist

$ sudo launchctl unload -w /Library/LaunchDaemons/org.macports.Privoxy.plist

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Privoxy.plist

$ sudo launchctl unload -w /Library/LaunchDaemons/org.macports.Polipo.plist

$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.Polipo.plist

$ nmap -p 3128,8118,8123 localhost

Starting Nmap 5.51 ( http://nmap.org ) at 2012-02-07 11:47 EST

Nmap scan report for localhost (127.0.0.1)

Host is up (0.00013s latency).

PORT STATE SERVICE

3128/tcp open squid-http

8118/tcp open privoxy

8123/tcp open polipo

Now web applications can use your filtering web proxy chain. If you use the config files below, websites will not know where you came from (HTTP_REFERER header is forged), and will not know your User Agent (also forged), and read access is block to several HTTP header fields. Ads are filtered. Your connection looks like this:

Application <--port 3128--> Squid <--port 8118--> Privoxy <----> Internet

Configure your network to add an option to route your web traffic through this proxy. System Preferences>Network>Wi-Fi/Ethernet/...>Locations:>Edit Locations...> Gear icon, Duplicate Location, Advanced...>Proxies> Check boxes for HTTP and HTTPS web proxies with proxy server localhost:3128.
While you're at it, configure your OS and browsers to block Adobe flash cookies. Read this WSJ article series to understand how this impacts your privacy.
- System Preferences>Flash Player>Block all sites from storing information, using your camera and microphone, and networking with peers. Also Delete all data and go to this Adobe Flash Player Settings web page and block all sites from storing information, using your camera and microphone, and networking with peers.
- Firefox/Safari>DO NOT ALLOW third party cookies, request not to be tracked
- Firefox Add-Ons: NoScript (blocks/manages JavaScript), Beef TACO (blocks/manages flash cookies), BetterPrivacy (blocks/manages flash cookies), and the EFFs HTTPS Everywhere.
- You can also download the Tor anonymous proxy chain for both OS X and iOS devices. This will run a little polipo proxy natively on mobile devices.

Here are the config file settings. Search through the config file too see the appropriate location for these settings. Turn off http_access and icp_access (squid), permit-access (privoxy), and allowedClients (polipo) if you do not want everyone on your LAN to be able to use the proxy. Double check that you're not running an open web proxy on the internet.

$ sudo vi /opt/local/etc/squid/squid.conf

# See http://www.privoxy.org/user-manual/config.html

# Define Privoxy as parent proxy (without ICP)

cache_peer 127.0.0.1 parent 8118 7 no-query

http_access allow localnet

icp_access allow localnet

via off

# old 'http_anonymizer standard'

header_access From deny all

# forge Referer in Privoxy

# header_access Referer deny all

header_access Server deny all

# forge User-Agent in Privoxy

# header_access User-Agent deny all

header_access WWW-Authenticate deny all

header_access Link deny all

# more privacy

header_access Cache-Control deny all

header_access Proxy-Connection deny all

header_access X-Cache deny all

header_access X-Cache-Lookup deny all

header_access Via deny all

header_access Forwarded-For deny all

header_access X-Forwarded-For deny all

header_access Pragma deny all

header_access Keep-Alive deny all

shutdown_lifetime 10 seconds

# See http://www.privoxy.org/user-manual/config.html

# Define ACL for protocol FTP

acl ftp proto FTP

# Do not forward FTP requests to Privoxy

always_direct allow ftp

# See http://www.privoxy.org/user-manual/config.html

# Forward all the rest to Privoxy

never_direct allow all

dns_nameservers 10.0.1.2 10.0.1.1

forwarded_for off

$ sudo vi /opt/local/etc/privoxy/config

forward / .

$ sudo vi /opt/local/etc/privoxy/match-all.action

{ \

+change-x-forwarded-for{block} \

+deanimate-gifs{last} \

+filter{refresh-tags} \

+filter{img-reorder} \

+filter{banners-by-size} \

+filter{webbugs} \

+filter{jumping-windows} \

+filter{ie-exploits} \

+hide-from-header{block} \

+hide-referrer{conditional-block} \

+session-cookies-only \

+set-image-blocker{pattern} \

}

/ # Match all URLs

# See http://www.christianschenk.org/blog/enhancing-your-privacy-using-squid-and-privo xy/

{ \

+hide-referrer{conditional-forge} \

+hide-user-agent{Mozilla/5.0} \

}

/ # Match all URLs

$ sudo vi /opt/local/etc/privoxy/user.action

# fix bing's travel site, others

{ -block }

ads1.msn.com/

.bing.com/travel/jsxc\.vjs\?

.onecause.com

.apple.com

.go.com

# sourceforge

{ -block -filter -deanimate-gifs}

.sourceforge.net

.dell.com

# expedia

{ -hide-user-agent }

.expedia.com

# don't filter downloads

{-filter -deanimate-gifs}

/.*\.iso(\?|$)

/.*\.mp3(\?|$)

/.*\.mp4(\?|$)

/.*\.mov(\?|$)

/.*\.mpg(\?|$)

/.*\.ogg(\?|$)

/.*\.aac(\?|$)

/.*\.zip(\?|$)

/.*\.pdf(\?|$)

/.*\.dmg(\?|$)

/.*\.tar(\?|$)

/.*\.gz(\?|$)

/.*\.dat(\?|$)

$ sudo vi /opt/local/etc/privoxy/config

proxyAddress = "0.0.0.0" # IPv4 only

allowedClients = 127.0.0.1, 10.0.1.0/16

Mac mini, Mac OS X (10.7.3), Lion Server, 8 GB

Posted on Feb 7, 2012 9:37 AM

Reply

Answer 1

tangucho

Level 1

0 points

Mar 27, 2012 11:22 AM in response to realzcubed

Awesome post! Thank you a lot!

I do beleive that if everyone should be concerned about privacy on the internet.

Can you give some guidelines about the same configuration but using:

Squidman (Squid GUI)

Tor Firefox browser bundle

I2P network

Freenet.

Maybe just using privoxy to forward ports properly? I am a beginner so any help would be great.

Thanks in advance!

Reply

Answer 2

realzcubed Author

Level 1

2 points

May 27, 2012 5:35 PM in response to realzcubed

Two comments:

1. Here's a proxy.pac file to use with iOS/other on your LAN. Bypasses the proxy for all LAN traffic, uses proxy for all other. Put this up on your local webserver, assuming dns resolves server.com to your server's LAN IP.

function FindProxyForURL(url, host)

{

if (

isPlainHostName(host) ||

shExpMatch(host, "192.168.*") ||

shExpMatch(host, "172.16.*") ||

shExpMatch(host, "127.*") ||

shExpMatch(host, "10.*") ||

dnsDomainIs(host, ".server.com") ||

dnsDomainIs(host, ".LOCAL") ||

dnsDomainIs(host, ".local") &&

!dnsDomainIs(host, ".server.com") ||

(url.substring(0,3) == "ftp")

)

return "DIRECT";

else

return "PROXY server.com:3128";

}

2. Comment out "head_access WWW-Authenticate deny all" to avoid breaking .htaccess web logins.

Reply

Answer 3

realzcubed Author

Level 1

2 points

May 27, 2012 5:41 PM in response to realzcubed

And this Privoxy setting will make all clients look like iPads, and probably has less Panopticlick entropy [https://panopticlick.eff.org/] than the generic User Agent given above:

+hide-user-agent{Mozilla/5.0 (iPad; U; CPU OS 4_3_5 like Mac OS X; en) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8L1 Safari/6533.18.5}

Reply

Answer 4

Franck P

Level 1

4 points

Aug 23, 2012 8:00 AM in response to realzcubed

This configuration looks great and I was try to apply for my laptop. Unfortunatly I'm not an expert, and I have problem with config file settings for squid.config.

I was installing squid (at first 2.7 version but later 3.1, because being able to use the GUI squidMan)), Privoxy and polipo with sucess with MacPorts. Using also MacPort to get nmap.and proxies look to be up :

Starting Nmap 6.01 ( http://nmap.org ) at 2012-08-23 21:59 PHT

Nmap scan report for localhost (127.0.0.1)

Host is up (0.00046s latency).

PORT STATE SERVICE

3128/tcp open squid-http

8118/tcp open privoxy

8123/tcp open polipo

Configure the network was not a problem (just an interrogation about FTP proxy ?)

To edit and add lines and save match-all.action,user.action

was fine also. I don't know why the command sudo vi /opt/local/etc/privoxy/config is repeat twice one to add forward / and later

proxyAddress = "0.0.0.0" # IPv4 only

allowedClients = 127.0.0.1, 10.0.1.0/16

I was add these 3 lines anyway, the main problem being I guess to put properly configurations for squid.conf

Here below the template gave by SquidMan,(easier for me getting the main lines!) I just have modified Privoxy as parent proxy but I was not able to manage properly where adding these settings.( getting error about localhost ie).

Could you kindly past them in this template ? I guess it will fix my configuration ! thank you in advance.

Sincerly,

Franck

# ----------------------------------------------------------------------

# WARNING - do not edit this template unless you know what you are doing

# ----------------------------------------------------------------------

# the parent cache

cache_peer 127.0.0.1 parent 8118 7 no-query no-digest no-netdb-exchange default

# disk and memory cache settings

cache_dir ufs %CACHEDIR% %CACHESIZE% 16 256

maximum_object_size %MAXOBJECTSIZE%

# store coredumps in the first cache dir

coredump_dir %CACHEDIR%

# the hostname squid displays in error messages

visible_hostname %VISIBLEHOSTNAME%

# log & process ID file details

cache_access_log %ACCESSLOG%

cache_log %CACHELOG%

cache_store_log %STORELOG%

pid_filename %PIDFILE%

# Squid listening port

http_port %PORT%

# Access Control lists

acl localhost src 127.0.0.1/32

acl to_localhost dst 127.0.0.0/8 0.0.0.0/32

acl manager proto cache_object

acl SSL_ports port 443

acl Safe_ports port 80 # http

acl Safe_ports port 21 # ftp

acl Safe_ports port 443 # https

acl Safe_ports port 70 # gopher

acl Safe_ports port 210 # wais

acl Safe_ports port 1025-65535 # unregistered ports

acl Safe_ports port 280 # http-mgmt

acl Safe_ports port 488 # gss-http

acl Safe_ports port 591 # filemaker

acl Safe_ports port 777 # multiling http

acl CONNECT method CONNECT

%ALLOWEDHOSTS%

%DIRECTHOSTS%

# Only allow cachemgr access from localhost

http_access allow manager localhost

http_access deny manager

# Deny requests to certain unsafe ports

http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports

http_access deny CONNECT !SSL_ports

# protect web apps running on the proxy host from external users

http_access deny to_localhost

# rules for client access go here

http_access allow localhost

%HTTPACCESSALLOWED%

# after allowed hosts, deny all other access to this proxy

# don't list any other access settings below this point

http_access deny all

# specify which hosts have direct access (bypassing the parent proxy)

%ALWAYSDIRECT%

always_direct deny all

# hierarchy stop list (squid-recommended)

hierarchy_stoplist cgi-bin ?

# refresh patterns (squid-recommended)

refresh_pattern ^ftp: 1440 20% 10080

refresh_pattern ^gopher: 1440 0% 1440

refresh_pattern -i (/cgi-bin/|\?) 0 0% 0

refresh_pattern . 0 20% 4320

Reply

Answer 5

Sep 25, 2012 9:44 PM in response to realzcubed

Hello,

This is way over my head and got to your post by searching "X-Forwarded-For" which I saw being used on a FireFox browser to conceal the geographical location when login in a website outside the US. I am looking for that same capability using Safari.

Will this script accomplish this among onther things? If so can you choose which IP address you "appear as"?

Thank you

Reply

Answer 6

Feb 11, 2013 2:27 PM in response to Cachibache

can you choose which IP address you "appear as"?

No -- you must use tor or a VPN tunnel for that. If the remote server doesn't know the ip you're attached to, its packets will never be routed back to you.

Reply

Answer 7

Feb 11, 2013 2:37 PM in response to realzcubed

This setup breaks a recent configuration change in Photo Stream. I had to break out wireshark to figure it out. The following addition to Privoxy's user.action file fixes Photo Stream (as well as automated TV Guide listings downloaded by EyeTV). I didn't attempt to minimize the block removals, but this works.

$ sudo vi /opt/local/etc/privoxy/user.action

# fix icloud, photostream

{ -block -filter -hide-user-agent -deanimate-gifs -hide-from-header -set-image-blocker }

TAG:^User-Agent: PhotoStreamAgent/

.apple.com

.icloud.com

.amazonaws.com

# TV Guide listings on EyeTV

{ -hide-user-agent }

.macrovision.com

Also, use this User-Agent for an iOS 6.1 passport:

$ sudo vi /opt/local/etc/privoxy/match-all.action

{ \

+hide-referrer{conditional-forge} \

+hide-user-agent{Mozilla/5.0 (iPad; CPU OS 6_1 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B141 Safari/8536.25} \

}

/ # Match all URLs

Reply