Squid proxy category blocklists: dstdomain, SquidGuard and SNI rules
Squid still sits at the edge of many school, campus, ISP and company networks, and inside a good number of firewall products. This guide is for the admins and product teams who need it to block by category: the three ways to match domains, how to build one file per category, HTTPS without decryption, and safe updates.
Three ways to block by category in Squid
Which one fits depends on how traffic reaches Squid and how large your lists are.
Domain lists in Squid itself
One file per category, loaded by an access control list. Matches the host name of plain web requests and of HTTPS CONNECT requests.
Best for explicit proxies and lists up to a few hundred thousand entries.An external rewriter
Squid hands each request to SquidGuard, which checks category folders compiled into database files and redirects blocked requests to a block page.
Best for very large lists split into many categories.Server name rules for intercepted HTTPS
With transparent interception, Squid can peek at the server name in the TLS handshake and terminate connections to blocked names.
Best where browsers are not configured to use the proxy.One categorized file that becomes every Squid list you need
Licence holders receive our database as CSV or Parquet with the categories of every domain in one field. Splitting it gives one Squid list per category, from the same 58-category taxonomy, refreshed with each delivery.
- One file per category for dstdomain or SquidGuard folders.
- Daily new domains to append between full refreshes.
- Runs entirely inside your network, with no outside lookups.
dstdomain lists: the details that matter
A dstdomain list is the quickest way to block by category. Four details decide whether it behaves as you expect.
# /etc/squid/cats/gambling.txt .bet365.com .pokerstars.com # the leading dot matches the domain # and every sub-domain # squid.conf acl cat_gambling dstdomain "/etc/squid/cats/gambling.txt" http_access deny cat_gambling # apply changes without a restart squid -k reconfigure
Five domains and the category files they would be written to
Real rows from our data. A domain with two categories goes into both files, so a rule on either category catches it.
| Domain | Categories in our data | Squid list files |
|---|---|---|
| pokerstars.com | Gambling | gambling.txt |
| hulu.com | Streaming Media | streaming.txt |
| steamcommunity.com | Gaming, Social Networking | gaming.txt and social.txt |
| deviantart.com | Arts & Entertainment, Community Sites | arts.txt and community.txt |
| imgur.com | Community Sites, Cloud Storage / File Hosting | community.txt and filehosting.txt |
Building one file per category from the database
The category field holds every category of a domain, separated by a pipe. A short script reads the file once and writes a list for each category you block.
# split_categories.py: one dstdomain list per category
import csv, re
wanted = {"Gambling": "gambling", "Adult": "adult",
"VPN / Proxy / Anonymizers": "vpn"}
out = {name: open(f"/etc/squid/cats/{name}.txt.new", "w") for name in wanted.values()}
with open("web_filtering_database.csv", newline="", encoding="utf-8") as f:
for row in csv.DictReader(f):
for cat in row["Web Filtering Category"].split("|"):
if cat in wanted:
out[wanted[cat]].write("." + row["Domain"] + "\n")
for fh in out.values():
fh.close()
For SquidGuard, write the same domains without the leading dot into a folder per category, such as gambling/domains, then compile them. The guide to the database specification lists every field in the file.
SquidGuard: category folders for very large lists
SquidGuard keeps each category in its own folder and compiles the lists into database files that load quickly. A minimal configuration:
# squidGuard.conf
dbhome /var/lib/squidguard/db
dest gambling { domainlist gambling/domains }
dest adult { domainlist adult/domains }
acl {
default {
pass !gambling !adult all
redirect http://blockpage.example/blocked
}
}
# compile the lists, then point Squid at it
squidGuard -C all
# squid.conf
url_rewrite_program /usr/bin/squidGuard
HTTPS without decryption
Category blocking by domain does not need to read encrypted content. How Squid sees the name depends on how traffic arrives.
| How traffic reaches Squid | What Squid sees | How to block by category |
|---|---|---|
| Explicit proxy (browser or system set to use it) | A CONNECT request with the host name and port | dstdomain lists match the CONNECT host |
| Transparent interception of port 443 | A TLS handshake with the server name | ssl_bump peek at step one, then ssl::server_name lists with ssl_bump terminate |
| Full decryption | The whole request | Possible, but needs a trusted certificate on every device and exemptions for banking and health |
# intercepted HTTPS: block by server name without decrypting acl step1 at_step SslBump1 acl cat_gambling_sni ssl::server_name "/etc/squid/cats/gambling.txt" ssl_bump peek step1 ssl_bump terminate cat_gambling_sni ssl_bump splice all
Splicing everything else leaves the content encrypted end to end. For what to decrypt when you do inspect, see SSL inspection and URL categories.
List size and the right method
Squid holds dstdomain lists in memory and rebuilds them on every reconfigure, so size decides the method more than anything else.
| Typical list | Rough size | Method that fits |
|---|---|---|
| A few safety categories from the most popular domains | Thousands to tens of thousands of entries | dstdomain, reloaded in a moment |
| A family set of 12 categories across the top 20 million domains | About 921,000 domains | dstdomain with enough memory, or SquidGuard |
| Many categories across the full database | Millions of entries | SquidGuard or another external helper |
The family-set figure comes from our own count of the 20 million most visited domains; see what the web is made of for the category shares.
Updating lists without interrupting users
Lists change every day. Four steps make an update a non-event.
Write beside the old files
Generate the new lists as .new files next to the live ones.
Check them
Compare line counts with the last run and test a few known domains in each.
Swap in one move
Rename the new files over the old ones, so Squid never reads a half-written list.
Reload
Run squid -k reconfigure, or compile and reload for SquidGuard.
How the database keeps Squid lists current
About 300,000 newly registered domains are categorized every day and delivered as daily delta files on plans that include them, and the full file is refreshed on your plan's schedule, from quarterly to daily. Append the deltas to your category lists between refreshes and rebuild everything when the full file arrives.
Six mistakes with Squid blocklists
Each of these has broken a Squid filter somewhere this year.
Forgetting the leading dot
Without it, only the exact name is blocked and every sub-domain stays open.
Deny rules below the allow
Squid stops at the first match. A deny placed after "allow localnet" never runs.
Overlapping entries
Parents and their sub-domains in one list produce warnings and slow reloads.
Not compiling SquidGuard
Updated text lists do nothing until the database files are rebuilt.
Decrypting everything
Blocking by domain needs only the name. Full decryption adds certificates, risk and complaints.
Lists that never change
New gambling and proxy domains appear daily; a list from last year misses them.
Squid lists from public blocklists, or from a licensed category file
The proxy configuration is the same; the input is what differs.
| For a Squid deployment | Public blocklists | Licensed category file |
|---|---|---|
| Categories | Whatever each list's maintainer chose | 58 categories with one definition each |
| A domain in two categories | In one list or none | Written to every matching category file |
| New domains | When someone reports them | About 300,000 categorized every day |
| Inside a product you sell | Licence differs list by list | OEM licence for embedding |
| Format | Converted from each list's own shape | One CSV or Parquet file to split |
Squid category blocking: frequently asked questions
How do I block a list of domains in Squid?
Put the domains in a file, one per line with a leading dot so sub-domains match, declare it with acl name dstdomain "/path/file", and add http_access deny name above your allow rules. Then run squid -k reconfigure.
Can Squid block HTTPS sites without decrypting them?
Yes. Browsers configured to use the proxy send a CONNECT request with the host name, which dstdomain rules can match. For intercepted traffic, Squid can peek at the server name in the TLS handshake and terminate connections to blocked names without decrypting the content.
What is the difference between dstdomain and SquidGuard?
dstdomain lists are loaded into Squid's memory and suit small and medium lists. SquidGuard is an external program that Squid calls for each request; it reads category folders compiled into database files, which suits very large lists split by category.
Why does Squid warn about my blocklist?
Usually because the list contains a domain and one of its sub-domains, for example .example.com and www.example.com. With a leading dot the parent entry already covers the sub-domain, so remove the duplicate.
How do I update category lists without interrupting users?
Write the new files beside the old ones, check them, swap them in with a rename, and reload Squid with squid -k reconfigure. For SquidGuard, compile the new database files before switching.
Can Squid show a block page for HTTPS sites?
Only partly. Without decryption, Squid can refuse a CONNECT request with an error, which most browsers show as a connection failure rather than your page. A readable block page for HTTPS needs decryption with a trusted certificate on every device.
Should I block by category or by URL path in Squid?
By category for most rules, because a domain list keeps working as sites add pages. URL path rules suit a few exceptions, such as one section of a large site, and need decryption for HTTPS.
Where do category lists for Squid come from?
From a categorized domain database. Licence holders of our database split it into one file per category, from 58 categories, and refresh the files with each delivery and with daily new domains.
Database licences start at $999 for the 1 million most popular domains, and OEM licences for proxy and firewall products are quoted per deployment. Tell us how many users the proxy serves and which categories you block, and we will suggest the tier.
Give your Squid proxy real category lists
Tell us where Squid runs, how traffic reaches it and how often you want lists refreshed. We will propose the files and the licence that fit.
Squid and SquidGuard are open source projects named here only to identify the software discussed. Web Filtering Database is not affiliated with either project.