Warning: Any example presented here is provided "as-is" with no support or guarantee of suitability. If you have any further questions about these examples please email the squid-users mailing list.
Squid-3.2 has now had experimental multi-process SMP support merged. It is designed to operate with a similar but different configuration to these while reducing much of the complexity of process instance management.
Squid-3.1 and older do not scale very well to Multi-CPU or Multi-Core systems. Some of its features do help, such as for example DiskDaemon, or COSS, or the ability to delegate parts of the request processing to external helpers such as Authenticators or other auxiliary software. Still Squid remains to this day very bound to a single processing core model. There are plans to eventually make Squid able to effectively use multicore systems, but something may be done already, by using a fine-tuned MultipleInstances setup.
This setup has been designed with a recent version of Squid in mind. It has been tested with Squid-3.1, but it should work with Squid-3.0 and Squid-2.7 as-is as well. Squid-2.6 and earlier can be coaxed to work, but it will be harder to setup and maintain.
It is also geared at expert system-administrators. MultipleInstances is not easy to manage and run, and system integration depends on the specific details of the operating system distribution of choice.
The setup laid out in this configuration example aims at creating on a system multiple running squid processes:
- a 'front-end' process which does
- logging, delay pools etc.
- in-memory hot-object caching
- load-balancing of the backend processes
- redirection etc.
- a 'back-end' processes farm, whose each does
- disk caching
- do the network heavy lifting
While this setup is expected to increase the general throughput of a multicore system, the benefits are anyways constrained, as the frontend process is still expected to be the bottleneck. Should anyone put this in production, he's encouraged to share the results to help others evaluate the effectiveness of the solution.
Squid Configuration File
For a 2-backends system, there are 5 configuration files to be used. You can click below each file on its filename to download it, no need to copy and paste. The .txt extension an artifact, please remove it.
This file contains the ACL's that are common to all running instances. This allows to change cluster-wide parameters without needing to touch each instance's. Each instance will still have to be reconfigured individually.
1 acl manager proto cache_object 2 acl localhost src 127.0.0.1/32 3 acl to_localhost dst 127.0.0.0/8 4 acl localnet src 192.168.0.0/24 5 acl SSL_ports port 443 6 acl Safe_ports port 80 # http 7 acl Safe_ports port 21 # ftp 8 acl Safe_ports port 443 # https 9 acl Safe_ports port 70 # gopher 10 acl Safe_ports port 210 # wais 11 acl Safe_ports port 1025-65535 # unregistered ports 12 acl Safe_ports port 280 # http-mgmt 13 acl Safe_ports port 488 # gss-http 14 acl Safe_ports port 591 # filemaker 15 acl Safe_ports port 777 # multiling http 16 acl CONNECT method CONNECT 17 18 19 20 cachemgr_passwd somepassword all
common backend parameters
Backends share most of the configuration, it makes sense to also join those
1 #you want the backend to have a small cache_mem 2 cache_mem 4 MB 3 4 refresh_pattern ^ftp: 1440 20% 10080 5 refresh_pattern ^gopher: 1440 0% 1440 6 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 7 refresh_pattern . 0 20% 4320 8 9 shutdown_lifetime 3 second 10 #debug_options all,8 11 12 # add user authentication and similar options 13 http_access deny !Safe_ports 14 http_access deny CONNECT !SSL_ports 15 http_access allow localhost 16 http_access deny all
1 # acl are shared among instances 2 include /usr/local/etc/lab/common.acl.conf 3 4 http_port 3128 5 6 #add backends 7 cache_peer localhost parent 4001 0 carp login=PASS name=backend-1 8 cache_peer localhost parent 4002 0 carp login=PASS name=backend-2 9 10 #you want the frontend to have a significant cache_mem 11 cache_mem 512 MB 12 13 refresh_pattern ^ftp: 1440 20% 10080 14 refresh_pattern ^gopher: 1440 0% 1440 15 refresh_pattern -i (/cgi-bin/|\?) 0 0% 0 16 refresh_pattern . 0 20% 4320 17 18 shutdown_lifetime 3 second 19 #debug_options all,8 20 21 # change /tmp to your own log directory, e.g. /var/log/squid 22 access_log /var/log/squid/frontend.access.log 23 cache_log /var/log/squid/frontend.cache.log 24 pid_filename /var/log/squid/frontend.pid 25 26 # add user authentication and similar options 27 http_access allow manager localhost 28 http_access deny manager 29 http_access deny !Safe_ports 30 http_access deny CONNECT !SSL_ports 31 http_access allow localnet 32 http_access allow localhost 33 http_access deny all
1 # acl are shared among instances 2 include /usr/local/etc/lab/common.acl.conf 3 4 http_port 127.0.0.1:4001 5 visible_hostname backend-1 6 unique_hostname backend-1 7 cache_dir aufs /mnt/cache-1 10240 128 128 8 9 # change /var/log/squid to your own log directory 10 access_log /var/log/squid/backend-1.access.log 11 cache_log /var/log/squid/backend-1.cache.log 12 pid_filename /var/log/squid/backend-1.pid 13 14 include /usr/local/etc/lab/common.backend.conf
1 # acl are shared among instances 2 include /usr/local/etc/lab/common.acl.conf 3 4 http_port 127.0.0.1:4002 5 visible_hostname backend-2 6 unique_hostname backend-2 7 cache_dir aufs /mnt/cache-2 10240 128 128 8 9 # change /var/log/squid to your own log directory 10 access_log /var/log/squid/backend-2.access.log 11 cache_log /var/log/squid/backend-2.cache.log 12 pid_filename /var/log/squid/backend-2.pid 13 14 include /usr/local/etc/lab/common.backend.conf