Running multiple instances of Squid on a system

Running multiple instances of Squid on a system is not hard, but it requires the administrator to make sure they don't stomp on each other's feet, and know how to recognize each other to avoid forwarding loops (or misdetected forwarding loops).

Relevant squid.conf directives

• visible_hostname

  • you may want to keep this unique for troubleshooting purposes

• unique_hostname

  • if you don't change the visible_hostname and want your caches to cooperate, at least change this setting to properly detect forwarding loops

• http_port

  • either the various squids run on different ports, or on different IP addresses. In the latter case the syntax to be used is 1.2.3.4:3128 and 1.2.3.5:3128

• icp_port, snmp_port

  • same as with http_port. If you don need ICP and SNMP, just disable them by setting them to 0.

• access_log, cache_log

  • you want to have different logfiles for you different squid instances. Squid might even work when all log to the same files, but the result would probably be a garbled mess

• pid_filename

  • this file must be changed. It is used by squid to detect a running instance and to send various internal messages (i.e. squid -k reconfigure)

• cache_dir

  • make sure that no overlapping cache_dirs exist. Squids do not coordinate when accessing them, and shuffling stuff around each others' playground is a bad thing ^TM

• include

  • to reduce duplication mistakes break shared pieces of config (ACL definitions etc) out into separate files which include pulls into each of the multiple squid.conf at the right places.

Tips

The easiest way I found to manage multiple squids running on one single box was to:

• create a configuration file per instance

• write a small shell script (named squid-something) per instance, containing:

exec /usr/local/sbin/squid -f /usr/local/etc/squid-something.conf $@

(of course, relevant path changes may have to be applied).

And then just run them as you would with a single-install squid setup.

Load Balancing behind a single port with iptables

by Felipe Damasio, Eric Dumazet, Jan Engelhardt

The theory of operation is: It puts the new HTTP connection on the extrachain chain. There, it marks each connection with a sequential number. This marking is latter checked by the PREROUTING chain and forwards it a squid port depending on the mark.

So, the first connection will be sent to port 3127, the second to 3128, the third to 3129, and the fourth back to 3127 (cycling through the ports on an even distribution).

The full thread on netfilter-devel where this was developed is here: http://marc.info/?l=netfilter-devel&m=127483388828088&w=2

(watch the wrap, iptables rules are single lines)

N=3
first_squid_port=3127

iptables -t mangle -F
iptables -t mangle -X
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT

iptables -t mangle -N extrachain
iptables -t mangle -A PREROUTING -p tcp --dport 80 -m conntrack --ctstate NEW -j extrachain

for i in `seq 0 $((N-1))`; do
  iptables -t mangle -A extrachain -m statistic --mode nth --every $N --packet $i -j CONNMARK --set-mark $i
done

for i in `seq 0 $((N-1))`; do
  iptables -t mangle  -A PREROUTING -i eth0 -p tcp --dport 80 -m connmark --mark $i -j TPROXY --tproxy-mark 0x1/0x1  --on-port $((i+first_squid_port))
done