A documented sample from the Web Filtering Database: 49,000 unique domains across six popularity bands, every one of the 59 categories represented.
A static sample for research and testing. It is not a blocklist and not a substitute for continuously updated filtering data.
See how multi-purpose domains carry several labels, and how the category mix changes from popular to long-tail sites.
Test joins between DNS or proxy logs and a category reference before connecting a live feed.
Concrete, labelled examples for courses on content filtering, classification and network policy.
Domains are drawn from six bands of source rank. Small categories are protected within each band so they still appear.
Version 1.0.0. Also on Figshare.
web-filtering-research-sample-49000.csvThe sample, CSV, 4.9 MBweb-filtering-research-sample-49000.jsonlThe sample, JSON Lines, 10.6 MBcategory-summary.csvLabel and primary-category counts for all 59 categories, 2 KBweb-filtering-taxonomy.csvThe 59 categories with descriptions (CSV), 16 KBweb-filtering-taxonomy.jsonThe 59 categories with descriptions (JSON), 28 KBdataset-analysis.jsonBand allocations, stratum quotas and summary statistics, 15 KBREADME.mdDataset description, 4 KBDATA_DICTIONARY.mdField definitions, 2 KBSAMPLING_METHODOLOGY.mdHow the sample was drawn, 4 KBCITATION.txtHow to cite, 1 KBLICENSE.txtCC BY 4.0 licence, 1 KBCHANGELOG.mdVersion history, 1 KBmanifest.jsonFile sizes and checksums (JSON), 2 KBSHA256SUMSChecksums, 1 KBRelated open data: AI Agent Page-Type Statistics (page types across 40.8 million domains, for AI agent egress research); AI Tools Taxonomy and Statistics.
The complete database covers up to 120 million domains with scheduled refreshes, API access and OEM licensing. See the full web filtering database.