Content Filter Data for Browser Extensions
A browser extension is the quickest way to put a content filter in front of a user, and the most constrained. Rule limits, storage budgets and store review all shape how category data can be used.
This guide is for teams building filtering extensions for schools, families and businesses. It covers which categories to ship, how to fit them inside the browser's limits, and how to keep them fresh.
What an extension filter needs from category data
An extension sees every navigation in the browser, which makes it precise. It also runs inside a sandbox with strict limits. The data must be shaped to fit.
A compact local index
Extension storage and memory are limited. You need a version of the data cut to the most visited domains and a small set of category IDs.
Fast first decisions
The block must happen before the page paints. A local index answers in microseconds; a network call on every navigation does not.
Wide coverage behind it
120 million categorized domains, with the 20 million most visited covering 99.99% of web traffic, so the backend can answer what the local index misses.
Data updates outside releases
Store review for every list update is not workable. Data must flow through its own channel, separate from extension code.
Ranked multiple labels
Each domain can carry several categories ranked by how much of the site they describe, so strict and relaxed profiles can treat mixed sites differently.
AI tool coverage
Schools and businesses now ask for AI chat controls first. A set of 16,000+ AI tool domains in 18 categories, updated daily, supports fine-grained rules.
Working within browser rule limits
Modern extension platforms ask filters to declare blocking rules for the browser to apply, rather than inspecting each request in extension code. Those rules come with limits.
Rules for the hot set
Turn the most visited domains in always-blocked categories into declarative block rules. They work even before the extension's service worker wakes.
Best for: Adult, proxies, malware on locked profiles.Lookup on navigation
Listen for navigation events, check the domain against a local index in storage, and redirect the tab to a block page when policy says so.
Best for: switchable categories and per-user profiles.Backend for the long tail
On a local miss, send the domain to your backend, which holds the full file or calls the lookup API. Cache the answer locally.
Best for: rare and newly registered domains.Which categories matter in an extension filter
Extension filters usually split categories into a locked core and admin switches. This split is a common starting point for school, family and workplace editions. See every definition on the category list.
| Category | Treatment | School edition | Family edition | Workplace edition |
|---|---|---|---|---|
| Adult | Locked core | Block | Block | Block |
| VPN / Proxy / Anonymizers | Locked core | Block | Block | Block or warn |
| Malware / Phishing / Suspicious | Locked core | Block | Block | Block |
| Gambling | Locked core | Block | Block | Admin choice |
| Social Networking | Switch | Block in class hours | Parent choice | Allow or time window |
| Gaming | Switch | Block | Time limited | Block |
| Streaming Media | Switch | Restricted | Parent choice | Admin choice |
| AI / LLM Tools | Switch | Approved tools only | Parent choice | Approved tools only |
| Remote Access Tools | Switch | Block | Allow | Approved tools only |
| Cloud Storage / File Hosting | Switch | Approved tools only | Allow | Approved tools only |
| URL Shorteners | Switch | Resolve target | Resolve target | Resolve target |
Offline database, lookup API, or both
For extensions the answer is nearly always both, in layers. What changes between products is how big each layer is.
The most visited domains in locked categories, compiled into block rules within the browser's limits. Works instantly, even when the extension is idle.
A compact index of the most visited domains across all 59 categories with their ranked labels. Checked on each navigation against the active profile.
Answers from the backend for domains the index did not hold, kept for a set period so repeat visits stay local.
The full licensed database, updated by daily delta files, plus the included lookup API as a final fallback.
Update cadence: regular refreshes without store releases
Extension stores review code updates, and review can take time. Keep category data out of the package and refresh it from your own server.
What changes, and when
- About 300,000 newly registered domains are categorized every day, delivered as delta files
- New domains are checked again after a week and after a month
- All other domains are refreshed monthly
- The AI tools set is updated daily
How the extension picks it up
- Backend applies deltas daily to the full store
- Backend builds a small daily patch for the local index and rule set
- Extension fetches the patch on a schedule and on browser start
- Dynamic rules are updated in place; no new version needed
Integration steps
These steps take an extension from the licensed file to a working filter. They assume your product already has an admin console for policies.
Load the file on your backend
Import the CSV, Parquet or JSON file into your database, keyed by domain with ranked categories.
Define editions and profiles
Map categories to locked and switchable groups per edition. Keep the map in config the backend serves to the extension.
Build the rule set
Compile the hot set of locked-category domains into declarative rules sized to the limits the browser reports.
Build the local index
Cut the most visited domains with their top labels into a compact, versioned file the extension downloads.
Handle navigations
On each navigation, normalize to host and registered domain, check the index and cache, and apply the profile.
Add the miss path
Send unknown domains to the backend, then the lookup API. Apply a default rule while waiting.
Ship a clear block page
Show the category and a request-access link. Clear block pages cut support tickets more than any data tweak.
Schedule the update job
Apply daily deltas on the backend and publish patches. Alert when an extension fleet stops fetching them.
Bypass routes and how to close them
An extension only filters the browser it lives in. Buyers will ask how you handle the usual workarounds, so plan answers that combine data and deployment.
Removing the extension
On personal browsers, users can simply uninstall it.
Close it: force install through managed browser policy.Private windows
Extensions may be off in private mode unless allowed.
Close it: policy to disable private mode or allow the extension there.Another browser
A second browser has no extension at all.
Close it: block other browsers by device policy, or add DNS filtering.Web proxies
A proxy site hides the real destination behind its own domain.
Close it: lock the VPN / Proxy / Anonymizers category with daily updates.Translation and cache front ends
Some services render another site's content under their own domain.
Close it: restrict them on strict profiles and inspect the target URL.Remote browsers
Browser-in-a-browser and remote desktop sites stream an unfiltered session.
Close it: switch off Remote Access Tools on school and family editions.Performance and privacy checklist
Store reviewers, school IT teams and enterprise security reviewers look at the same handful of things. Check them on every release.
| Check | Why it matters | Target |
|---|---|---|
| Local index size in storage | Large downloads slow first run on low-end school laptops | Sized per edition, compressed, versioned |
| Navigation check time | Slow checks let the page flash before the block | Local answers well under one frame |
| Share of navigations needing the backend | Each miss is a network call and a privacy question | Small and stable after a warm-up period |
| Data sent to the backend | Reviewers reject extensions that ship browsing history | Domain only, no paths or query strings |
| Requested permissions | Broad permissions draw extra store review | Only what the filter actually uses |
| Update fetch success | Stale lists let new proxies through | Alert when a share of the fleet misses a day |
Extension filter types and their data needs
"Content filter extension" covers several different products. Each one leans on different parts of the category data and a different local index size.
| Product type | Main buyer | Data focus | Local index | Typical extras |
|---|---|---|---|---|
| School filter extension | District IT teams | Adult, proxies, games, social, AI tools, plus allow-listed learning sites | Large, since school laptops often roam off network | Class-hour schedules, reporting for safeguarding staff |
| Family filter extension | Parents | Adult, gambling, dating, proxies, age profiles | Medium | Time limits and weekly summaries |
| Workplace policy extension | IT and security teams | AI tools, file hosting, remote access, malware and phishing | Medium, with more security categories | Warn-and-continue pages, approved tool lists |
| Browsing safety extension | Consumers | Malware / Phishing / Suspicious and newly registered domains | Small, with heavy daily updates | Link checks in search results and webmail |
| Focus or productivity extension | Individuals and teams | Social, streaming, games, shopping, news | Small to medium | Focus sessions and usage reports |
Block pages, warnings and access requests
The block page is where users meet your category data. A good one explains the decision and offers a way forward; a bad one turns every edge case into a support ticket.
Name the category
Show which category triggered the block, in plain words. Users accept a clear reason far more readily than a generic error.
Offer a request button
Let users send the domain and a short reason to their admin or parent. Pre-fill the category so the reviewer sees it at once.
Warn instead of block
For workplace editions, a "continue anyway" page for borderline categories, with logging, keeps work flowing while recording risk.
Show mixed sites fairly
When a domain carries several categories, show the one that matched the rule, not the top-ranked one, so the reason makes sense.
Keep overrides local
An approved request should apply at once on the device, before the next data update, and sync to the admin console.
Never leak the list
Block pages should not reveal other domains in the same category or how many entries the local index holds.
Licensing data for a filtering extension
An extension you distribute to customers embeds the data, so it needs an OEM licence. The model is set out on the OEM licensing page.
OEM / embedded
The data runs inside your extension and backend. Users and admins see categories and block decisions, not the raw file.
Group-wide
For companies shipping an extension plus a DNS filter, an app or an appliance. One licence can cover the product family.
Protect the local index
Because the index sits on user devices, obfuscate it so it cannot be lifted as a list. Resale or publication of the raw data is not allowed.
Pricing
Starter database tiers from $999; enterprise and OEM licences are quoted per deployment. See pricing.
Browser extension filter FAQ
Can a browser extension hold a full URL category database?
Not as browser blocking rules, because rule counts are limited. Extensions usually hold a compact category index for the most visited domains in extension storage, check navigations against it, and ask a backend or the lookup API for the rest.
How many blocking rules can an extension use?
At the time of writing, Chromium-based browsers guarantee each extension 30,000 static rules, with access to a larger shared pool, and allow up to 30,000 dynamic rules for safe actions such as block and allow. Limits change, so check the current browser documentation.
Do we need to publish a new extension version for each data update?
No. Keep category data out of the extension package and download it from your own server into extension storage. Code changes go through store review; data updates do not need to.
Which categories matter most for an extension filter?
Adult, VPN / Proxy / Anonymizers, Malware / Phishing / Suspicious and Gambling are the usual locked core. Social Networking, Gaming, Streaming Media and AI / LLM Tools are common switchable categories for schools, families and workplaces.
How fresh is the data?
About 300,000 newly registered domains are categorized daily and delivered as delta files, rechecked after a week and after a month, and all other domains get regular monthly refreshes.
Can users get around an extension filter?
On unmanaged browsers they can remove the extension, use private windows or switch browsers. Products that need a firm block install the extension through managed browser policy and add a network or DNS layer.
Should the local index ship inside the extension package?
Ship a small starter index so the filter works on first run, then download the full local index from your server. That keeps the package small and lets you update data without a new store release.
How do we handle subdomains in an extension filter?
Check the full host first, then walk up to the registered domain. Subdomains inherit the parent's category unless they carry their own, which matters for large platforms that host very different content on different subdomains.
Can the same data power both the extension and a DNS filter?
Yes. The same licensed file feeds both. Export formats for licensed customers include hosts files, DNS RPZ and AdGuard/Pi-hole style lists, so a network layer can share categories with the extension.
What should the extension do when the backend is unreachable?
Keep filtering from the rule set, local index and cache. Apply the profile's default rule to unknown domains, queue them, and resolve them once the connection returns.
Do we need different data for each browser?
No. The category data is the same for every browser. What differs is how each browser lets extensions block requests and how many rules it allows, so the rule-building step on your backend produces one rule set per browser family from the same source file.
Build your extension filter on current category data
Tell us your editions, your user count and your browsers. We will help you size the local index and the licence.
License the database OEM licensing