Tuesday 5 November 2013

Squid 3.3.10 - Transparent Proxy for HTTP and HTTPS

Hey there,

for several years the squid proxy can be used as transparent proxy for HTTP and also HTTPS. As I was curious how it will work and how hard it is to setup, I've just installed and configured it.

First I installed a fresh virtual machine with Debian 7.2. In Debian you could either install Squid 2.7 or Squid 3.1 via apt-get (apt-get install squid or apt-get install squid3). Unfortunately to make a transparent proxy that also supports all HTTPS features, at least version 3.2 is needed. So I downloaded the latest sources (Version 3.3.10) directly from squid-cache.org. Before installing, the following packages should be installed in Debian, otherwise errors will pop-up during configure or make:
# apt-get install build-essential
# apt-get install libssl-dev
After unpacking the squid sources it is important to use the following configure statement, to activate ssl, because it is disabled by default:
#./configure --prefix=/usr/local/squid --enable-icap-client --enable-ssl --enable-ssl-crtd --with-default-user=squid
Afterwards you can compile and install squid:
# make all
# sudo make install
Now squid is installed in /usr/local/squid. As next step the user squid should be created and the log directory should be allocated to that user:
# useradd squid
# chown -R squid:squid /usr/local/squid/var/logs/
The next steps I've copied from the squid documentation (2): 
Afterwards you must create the swap directories. Do this by running Squid with the -z option:
# /usr/local/squid/sbin/squid -z
Once the creation of the cache directories completes, you can start Squid and try it out. Probably the best thing to do is run it from your terminal and watch the debugging output. Use this command:
# /usr/local/squid/sbin/squid -NCd1
If everything is working okay, you will see the line:
Ready to serve requests. 
If you want to run squid in the background, as a daemon process, just leave off all options:
# /usr/local/squid/sbin/squid
Now you should have a running squid on port 3128. But we still do not support HTTPS requests and the proxy is still not transparent. The next steps will be modifing squid.conf and put in some iptables rules. But at first we need to create our your own CA (Certificate Authority):
# cd /usr/local/squid
# mkdir ssl_cert
# cd ssl_cert
# openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout myCA.pem -out myCA.pem
This pem file can now be imported in your certificate store in your browser. Then you will not get any certificate errors when surfing HTTPS sites later via our transparent squid.
Next we need to replace the line "http_port 3128" with the following lines in /usr/local/squid/etc/squid.conf:
http_port 3128 intercept
https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/usr/local/squid/ssl_cert/myCA.pem
acl broken_sites dstdomain .example.com
ssl_bump none localhost
ssl_bump none broken_sites
ssl_bump server-first all
sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /usr/local/squid/var/lib/ssl_db -M 4MB
sslcrtd_children 5
Also ip-forwarding needs to be activated:
# echo "1" > /proc/sys/net/ipv4/ip_forward
Finaly we need to insert our iptables rules to redirect the traffic to squid:
# iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 3128
# iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 3127
# iptables -I INPUT -p tcp -m tcp --dport 3127 -j ACCEPT
Another folder needs to be created, for the dynamically generated certificates:
# mkdir /usr/local/squid/var/lib
# /usr/local/squid/libexec/ssl_crtd -c -s /usr/local/squid/var/lib/ssl_db -M 4MB
# chown -R squid:squid /usr/local/squid/var/lib/ssl_db/
Now you should start squid in debugging mode:
# /usr/local/squid/sbin/squid -NCd9
If the process is running and you get something similar to this, you work was successfull:
2013/11/04 22:39:16| Accepting NAT intercepted HTTP Socket connections at local=0.0.0.0:3128 remote=[::] FD 19 flags=41
2013/11/04 22:39:16| Accepting NAT intercepted SSL bumped HTTPS Socket connections at local=0.0.0.0:3127 remote=[::] FD 20 flags=41
To fully work as transparent HTTPS proxy, your clients in the network needs now the IP of this proxy as gateway address and the pem certificate needs to be imported in the browser of the clients. 

Now you can start squid by exeuting:
# /usr/local/squid/sbin/squid

Debugging:

If you have any problems you should check if squid and their ports are running. You can do this by using netstat:
# netstat -tulpen
You should then see port 3128 and 3127. If not execute "killall squid" several times and restart squid in debugging mode with 
# /usr/local/squid/sbin/squid -NCd9
You can also have a look at the access.log during browsing or via tcpdump to see if the packets are really arriving at your proxy. 

Hint:
This was just a quick'n'dirty how-to on how a transparent proxy supporting HTTPS can be created. This setup is for lab environments to get to know squid and it's capabilites and not for productive use. For example your private key is in the pem certificate which should be seperated from the certificate your deploying to your browser. 

Links
(1) Download Squid Source
(2) Installing Squid
(3) Dynamics SSL Certificate Generation
(4) SSL Bump

14 comments:

  1. I'm trying but no luck with squid 3.3 http and https.
    Is it possible to post a working squid.conf , please?

    ReplyDelete
  2. how to install the Squid 3.4.2 is running smoothly?, and the correct way how to install it

    ReplyDelete
  3. Just use these config settings:

    http_port 3128 intercept
    https_port 3127 intercept ssl-bump generate-host-certificates=on dynamic_cert_mem_cache_size=4MB cert=/usr/local/squid/ssl_cert/myCA.pem
    acl broken_sites dstdomain .example.com
    ssl_bump none localhost
    ssl_bump none broken_sites
    ssl_bump server-first all
    sslcrtd_program /usr/local/squid/libexec/ssl_crtd -s /usr/local/squid/var/lib/ssl_db -M 4MB
    sslcrtd_children 5

    ReplyDelete
    Replies
    1. https://www.facebook.com, https://www.twitter.com on browser say The page isn't redirecting properly

      Delete
  4. Is https working on this configuration?

    ReplyDelete
  5. Just installed and configured using deb 7.5 and squid 3.4 on one ethernet, squid was running, but when I try access some site, I get this error WARNING: Forwarding loop detected for:
    What causes it? Could you help me?

    ReplyDelete
  6. Worked fine with squid 3.4.5... Thanks!

    ReplyDelete
  7. Show me:

    FATAL: The ssl_crtd helpers are crashing too rapidly, need help!

    :(

    ReplyDelete
  8. This comment has been removed by a blog administrator.

    ReplyDelete
  9. This comment has been removed by a blog administrator.

    ReplyDelete
  10. This comment has been removed by a blog administrator.

    ReplyDelete
  11. This comment has been removed by a blog administrator.

    ReplyDelete
  12. This comment has been removed by a blog administrator.

    ReplyDelete
  13. Store logging disabled
    2014/09/23 19:25:28| Swap maxSize 0 + 262144 KB, estimated 20164 objects
    2014/09/23 19:25:28| Target number of buckets: 1008
    2014/09/23 19:25:28| Using 8192 Store buckets
    2014/09/23 19:25:28| Max Mem size: 262144 KB
    2014/09/23 19:25:28| Max Swap size: 0 KB
    2014/09/23 19:25:28| Using Least Load store dir selection
    2014/09/23 19:25:28| Set Current Directory to /usr/local/squid/var/cache/squid
    2014/09/23 19:25:28| Finished loading MIME types and icons.
    2014/09/23 19:25:28| HTCP Disabled.
    2014/09/23 19:25:28| Squid plugin modules loaded: 0
    2014/09/23 19:25:28| Adaptation support is off.
    2014/09/23 19:25:28| Accepting NAT intercepted HTTP Socket connections at local=[::]:3128 remote=[::] FD 19 flags=41
    2014/09/23 19:25:28| Accepting HTTP Socket connections at local=[::]:3128 remote=[::] FD 20 flags=9
    2014/09/23 19:25:28| Accepting NAT intercepted SSL bumped HTTPS Socket connections at local=[::]:3127 remote=[::] FD 21 flags=41
    2014/09/23 19:25:28| ERROR: listen( FD 20, [::] [ job2], 256): (98) Address already in use
    2014/09/23 19:25:29| storeLateRelease: released 0 objects

    ReplyDelete