Squid 3.3.10 - Transparent Proxy for HTTP and HTTPS

Hey there,

for several years the squid proxy can be used as transparent proxy for HTTP and also HTTPS. As I was curious how it will work and how hard it is to setup, I've just installed and configured it.

First I installed a fresh virtual machine with Debian 7.2. In Debian you could either install Squid 2.7 or Squid 3.1 via apt-get (apt-get install squid or apt-get install squid3). Unfortunately to make a transparent proxy that also supports all HTTPS features, at least version 3.2 is needed. So I downloaded the latest sources (Version 3.3.10) directly from squid-cache.org. Before installing, the following packages should be installed in Debian, otherwise errors will pop-up during configure or make:

# apt-get install build-essential
# apt-get install libssl-dev

After unpacking the squid sources it is important to use the following configure statement, to activate ssl, because it is disabled by default:

#./configure --prefix=/usr/local/squid --enable-icap-client --enable-ssl --enable-ssl-crtd --with-default-user=squid

Afterwards you can compile and install squid:

# make all
# sudo make install

Now squid is installed in /usr/local/squid. As next step the user squid should be created and the log directory should be allocated to that user:

# useradd squid
# chown -R squid:squid /usr/local/squid/var/logs/

The next steps I've copied from the squid documentation (2): 

Afterwards you must create the swap directories. Do this by running Squid with the -z option:

# /usr/local/squid/sbin/squid -z

Once the creation of the cache directories completes, you can start Squid and try it out. Probably the best thing to do is run it from your terminal and watch the debugging output. Use this command:

# /usr/local/squid/sbin/squid -NCd1

If everything is working okay, you will see the line:

Ready to serve requests. 

If you want to run squid in the background, as a daemon process, just leave off all options:

# /usr/local/squid/sbin/squid

Now you should have a running squid on port 3128. But we still do not support HTTPS requests and the proxy is still not transparent. The next steps will be modifing squid.conf and put in some iptables rules. But at first we need to create our your own CA (Certificate Authority):

# cd /usr/local/squid
# mkdir ssl_cert
# cd ssl_cert
# openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout myCA.pem -out myCA.pem

This pem file can now be imported in your certificate store in your browser. Then you will not get any certificate errors when surfing HTTPS sites later via our transparent squid.
Next we need to replace the line "http_port 3128" with the following lines in /usr/local/squid/etc/squid.conf:

http_port 3128 intercept
https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/usr/local/squid/ssl_cert/myCA.pem
acl broken_sites dstdomain .example.com
ssl_bump none localhost
ssl_bump none broken_sites
ssl_bump server-first all
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /usr/local/squid/var/lib/ssl_db -M 4MB
sslcrtd_children 5

Also ip-forwarding needs to be activated:

# echo "1" > /proc/sys/net/ipv4/ip_forward

Finaly we need to insert our iptables rules to redirect the traffic to squid:

# iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
# iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3127
# iptables -I INPUT -p tcp -m tcp --dport 3127 -j ACCEPT

Another folder needs to be created, for the dynamically generated certificates:

# mkdir /usr/local/squid/var/lib
# /usr/local/squid/libexec/ssl_crtd -c -s /usr/local/squid/var/lib/ssl_db -M 4MB
# chown -R squid:squid /usr/local/squid/var/lib/ssl_db/

Now you should start squid in debugging mode:

# /usr/local/squid/sbin/squid -NCd9

If the process is running and you get something similar to this, you work was successfull:

2013/11/04 22:39:16| Accepting NAT intercepted HTTP Socket connections at local=0.0.0.0:3128 remote=[::] FD 19 flags=41
2013/11/04 22:39:16| Accepting NAT intercepted SSL bumped HTTPS Socket connections at local=0.0.0.0:3127 remote=[::] FD 20 flags=41

To fully work as transparent HTTPS proxy, your clients in the network needs now the IP of this proxy as gateway address and the pem certificate needs to be imported in the browser of the clients. 

Now you can start squid by exeuting:

# /usr/local/squid/sbin/squid

Debugging:

If you have any problems you should check if squid and their ports are running. You can do this by using netstat:

# netstat -tulpen

You should then see port 3128 and 3127. If not execute "killall squid" several times and restart squid in debugging mode with 

# /usr/local/squid/sbin/squid -NCd9

You can also have a look at the access.log during browsing or via tcpdump to see if the packets are really arriving at your proxy. 

Hint:

This was just a quick'n'dirty how-to on how a transparent proxy supporting HTTPS can be created. This setup is for lab environments to get to know squid and it's capabilites and not for productive use. For example your private key is in the pem certificate which should be seperated from the certificate your deploying to your browser. 

Links
(1) Download Squid Source
(2) Installing Squid
(3) Dynamics SSL Certificate Generation
(4) SSL Bump