webfilteringdatabase.com
Home Find Your Solution
Features
Domain Categorization API Real-Time Classification 59 Filtering Categories Offline Database (120M) Managed DNS Filter ML Classification Content Classification
Industries
K-12 Schools Corporate Healthcare Government ISPs
Tools
Domain Lookup Bulk Categorization Category Explorer
Resources
Pricing API Documentation Login / Sign Up

ISP-level content filtering

ISP-level content filtering puts category-based protection inside the provider's network, so every device on a subscriber's line is covered without apps or settings at home. It is how many operators deliver family safety and security add-ons at scale.

This guide is for operators, carrier engineers and product managers. It covers where filtering can sit, how to tie policy to subscribers, how to package the service and what the category data must deliver.

120Mcategorized domains for subscriber policy
59web filtering categories, plus the IAB taxonomy
~300knewly registered domains categorized per day
99.99%of web traffic covered by the top 20M domains

Three places a provider can filter

"ISP filtering" can mean filtering in the core network, in the home router the ISP supplies, or in an app the ISP offers. Each reaches a different set of devices and needs a different build.

Network level

In the provider's resolvers or gateways

  • Covers every device on the line
  • No customer equipment changes
  • Policy per line or account
  • Bypassed by outside resolvers and VPNs
CPE level

In the router the ISP supplies

  • Per-device and per-profile rules at home
  • Schedules and pause buttons
  • Needs firmware support and category data on the box
  • See router filtering for OEMs
Device level

In an ISP-branded app

  • Follows phones and laptops outside the home
  • Richest controls for parents
  • Needs installation per device
  • Often bundled with mobile plans
QuestionNetwork levelCPE levelDevice level
Who must install something?NobodyNobody, if the router supports itThe subscriber, on each device
Per-child or per-device rulesLimited, usually per lineYesYes
Works outside the homeNo, unless an encrypted endpoint is usedNoYes
Time to roll out to the baseFast, one central changeDepends on firmware updates and router ageSlow, depends on app uptake
Where category data livesProvider resolversOn the router or in its cloudIn the app or its cloud

Most operators start at network level because it is the fastest and simplest way to protect every subscriber who opts in. The CPE and app layers then add detail for households that want it. The difference between domain-level and URL-level controls is explained in DNS filtering vs web filtering.

Reference architecture for network-level filtering

The core of most ISP filtering services is a resolver cluster with a policy engine and local category data. The pieces below appear in nearly every design, whatever software is used. Many operators already run most of them for ordinary DNS and only add the policy and data layers.

Subscriber identity

Maps each query to a subscriber and their chosen profile, using the line's address, session data from the access network, or a per-subscriber encrypted DNS endpoint.

Anycast resolvers

Filtering resolvers spread across points of presence, reached through the same addresses, so load and failures are absorbed without subscriber changes.

Category data in memory

The domain category database loaded next to each resolver, so the lookup adds negligible delay and no query leaves the network.

Policy engine

Profiles defined as sets of categories, plus subscriber allow and block entries, evaluated for each query.

Block response

An empty answer or an address for an explanation page. For HTTPS, many operators prefer silent blocks to avoid certificate warnings.

Self-service portal

Where subscribers turn the service on, pick a profile, add exceptions and see simple reports.

Data pipeline

Daily delta files for new domains and a monthly full refresh, loaded without restarting resolvers.

Fixed, mobile and wholesale networks

NetworkHow subscribers are identifiedTypical service
Fixed broadbandLine or session data from the access network, or the public address where it is uniqueHousehold family and security profiles
MobileSubscriber session data in the mobile core, usually before address translationAge-appropriate defaults on child and prepaid plans, security on all
Wholesale and virtual operatorsIdentity passed from the host network or a per-brand resolver addressEach brand chooses its own profiles on a shared platform
Business connectivityDedicated address ranges per customerSecurity baseline, optional content rules managed by the customer

Groups with several brands or countries often prefer one platform and one category licence across all of them. Group-wide licences cover that case without separate contracts per brand.

The CGNAT problem

Where many subscribers share one public address through carrier-grade NAT, the resolver may only see the shared address. Policy must then come from inside the access network, before translation, or from a per-subscriber encrypted DNS endpoint that identifies the account directly.

Capacity and performance planning

Memory, not calls

Hold the category data in memory on each resolver node. A hash lookup per query keeps added latency well below what subscribers can notice.

Cache wisely

Cache the category result per domain, separate from the DNS answer, so profile changes do not wait for answer caches to expire.

Hot reloads

Apply daily delta files without restarts. Swap the full monthly set node by node behind anycast.

Headroom

Plan for peaks such as major sports events and new device launches, when lookup volume jumps for hours.

Design tip: keep filtered and unfiltered subscribers on the same resolver cluster with different policies, rather than separate clusters. It simplifies capacity planning and lets subscribers switch profiles instantly.

Packaging the service for subscribers

Operators usually offer two or three named options. Clear names and short category lists sell better than long settings pages. Subscribers can then add or remove single categories if they want more control.

Security

For every subscriber who wants protection without content rules.

  • Malware / Phishing / Suspicious
  • Newly registered domain caution
  • Botnet and command-and-control domains
  • Often included at no extra cost to the plan

Family

For households with children.

  • Everything in Security
  • Adult, Gambling, Dating, Illegal Drugs
  • Weapons, Violence, Hate/Discrimination
  • VPN / Proxy / Anonymizers
  • Safe search enforcement

Business

For small business lines.

  • Everything in Security
  • Adult and Gambling blocks
  • Optional social and streaming limits
  • Remote Access Tools control
  • Monthly category report

Why operators offer filtering at all

Household trust

Parents look for help with online safety. A simple family option answers a real worry at sign-up and renewal.

Fewer infected lines

Blocking malware domains for all subscribers reduces infected devices, abuse complaints and outbound attack traffic.

Small business bundles

A security and content add-on makes business plans stand out without new hardware on site.

Public expectations

In many markets, regulators and the public expect providers to make family protection easy to find and use.

Each category name comes from our web filtering categories list. For individual category detail, see gambling, adult content and malware.

Opt-in, default-on and the law

Whether filtering is offered, switched on by default or required depends on the market, and sometimes on the type of plan. This is the area where operators most need current legal advice. The points below are general and not a substitute for it.

ModelHow it worksPoints to consider
Opt-inSubscriber turns filtering on and chooses a profileGenerally the lowest regulatory risk; uptake depends on marketing and ease of activation
Active choiceNew subscribers are asked to choose on or off at sign-upHigher uptake, needs a clear, neutral question
Default-on, easy offFamily filter active unless the account holder turns it offUsed in some markets for consumer lines; check whether local rules allow it
Legal blocking ordersSpecific sites blocked because a court or authority requires itSeparate from category filtering; handle through a dedicated list and process

Open internet rules

In the EU, Regulation (EU) 2015/2120 limits blocking by providers. Regulator guidance has discussed filtering that end users choose and control as a separate case. Check current guidance in each country.

Data protection

Lookup logs tied to subscribers are personal data in many jurisdictions. Minimise retention and keep reporting aggregated where you can.

Terms of service

Describe what the filter does, what it cannot guarantee, and how to request a review of a block.

Child safety rules

Some countries set expectations for online child safety that touch providers. Requirements change, so review them with counsel each year.

What the category data must deliver at carrier scale

Operators run billions of lookups a day. At that scale, small gaps in data become thousands of complaints or missed threats. These are the requirements we hear from network teams.

A useful test before choosing data: take a day of anonymized lookup volume by domain, and check what share of it gets a confident category. Weighted by traffic, not by domain count, that number predicts how the service will feel to subscribers.

Coverage of real traffic

The 20 million most visited domains cover 99.99% of web traffic and usage. Our database categorizes 120 million domains, so the long tail is also labelled.

New domains fast

About 300,000 newly registered domains are categorized every day and checked again after a week and after a month, as content changes.

Runs inside the network

Delivered as CSV, Parquet or JSON files that load into your own resolvers, on-premise and air-gapped if needed. No per-query calls leave the network.

Carrier formats

Licensed customers can export DNS RPZ and other list formats. See blocklist formats for options.

Ranked categories

Several categories per domain, ordered by how much of the site each describes, so a news portal with a betting section is not treated as a casino.

Measured accuracy

Regular manual validations measure accuracy, with validation reports available to licence holders.

Terms operators use

RPZ

Response Policy Zone, a standard way to feed a resolver the names it should rewrite or block.

Clean pipe

A connection with security filtering applied in the network, usually sold to business customers.

Profile

A named set of blocked categories, such as Family or Security, assigned to a subscriber.

Delta file

A daily file with new and changed domain categories, applied on top of the full set.

The ISP web filtering and telecommunications web filtering pages cover sector needs. Our earlier overview of ISP content filtering looks at carrier-grade data requirements.

Bypass, encrypted DNS and what subscribers should know

A network filter protects devices that use the provider's resolver. Subscribers, or their children, can try to step around it. Honest messaging and a few design choices handle most cases. Parents generally accept limits when they are explained up front, and lose trust when they discover them on their own.

Bypass routeProvider response
Device set to a public resolverFor family profiles, offer router settings that force DNS, and explain the limit in the portal
Browser encrypted DNS to an outside providerOffer your own filtered DoH and DoT endpoints, and answer the browser canary domain for filtered lines
VPN or proxy appsBlock the VPN / Proxy / Anonymizers category in family profiles
Mobile data on a different networkBundle the same profile on mobile lines or in the ISP app

The full picture is in DNS over HTTPS and web filtering. For a household, network filtering is a strong baseline, and device controls add the final layer.

Pitfalls operators run into

Promising more than DNS can do

Marketing that implies complete protection leads to complaints when a VPN or mobile data gets around it. Describe the service as a strong first layer.

Block pages that look like attacks

HTTPS block pages without a trusted certificate trigger browser warnings. Silent blocks with a portal explanation cause fewer calls.

Profiles nobody understands

Twenty toggles on a settings page reduce activation. Three named profiles with optional extras work better.

Stale category data

If new adult and scam domains are not labelled quickly, parents notice. Daily updates are the minimum for a consumer service.

Launching an ISP filtering service

A typical launch runs in four stages. Each one has a clear exit test before the next starts, such as a target for wrong-block reports or activation completion.

  1. Lab

    Load category data into a test resolver, define the profiles, and measure query latency with and without policy.

  2. Staff pilot

    Turn the service on for employee lines. Collect wrong-block reports and tune profiles.

  3. Soft launch

    Offer opt-in to a region or segment. Watch support volume, uptake and activation drop-off.

  4. General availability

    Market the service, add it to sign-up flows, publish the review process and report usage monthly.

Marketing that sets the right expectations

Describe what the service blocks in everyday words, such as adult sites, gambling and scam pages. Show the three profiles side by side. Say clearly that it works on the home connection and that apps on each device can add more control.

Put the activation switch where subscribers already go: the account app, the router page and the sign-up flow. Every extra step lowers uptake, and most parents decide in the first visit.

Support readiness

Front-line scripts

Agents need a lookup tool to see a domain's category and the subscriber's profile in one view.

Review requests

A simple form for subscribers to ask for a category review, routed to a small specialist queue.

Instant changes

Profile switches should apply within minutes. Short TTLs on block answers help.

Subscriber reports

A short monthly summary of blocked threats builds trust and reduces churn from the add-on.

How to measure success

MeasureWhy it matters
Activation rate among eligible subscribersShows whether the offer is easy to find and understand
Wrong-block reports per thousand active linesThe clearest signal of data quality and profile fit
Threat blocks per line per monthEvidence of value for subscriber reports and marketing
Share of lookups with no categoryShould be small and falling, otherwise coverage is weak
Deactivation rate after the first monthHigh numbers point to overblocking or confusing settings

Support teams can use the domain lookup tool to check a category, and the lookup API for integration into agent desktops.

Frequently asked questions

What is ISP-level content filtering?

Filtering inside the provider's network, usually in its DNS resolvers or a gateway, so subscribers are protected without installing anything.

How do ISPs filter content?

Mostly with filtering resolvers that check each lookup against the subscriber's profile, tied to the subscriber through line, address or account.

Is ISP content filtering opt-in or default?

It depends on the market and provider. Opt-in is common; some markets have used default-on with easy opt-out. Check the current law.

Does network-level filtering conflict with net neutrality?

Rules differ by jurisdiction. In the EU, subscriber-chosen filtering has been treated differently from provider-imposed blocking. Take legal advice for each market.

Can subscribers bypass ISP filtering?

Yes, with other resolvers, encrypted DNS or VPNs. Offer filtered encrypted endpoints and block anonymizers in family profiles.

What category data does an ISP need?

Broad coverage, fast new-domain categorization, clear family and security categories, local delivery and carrier formats such as RPZ.

How is your data licensed for operators?

Under internal or OEM licences, including group-wide licences for operators with several brands. Starter database tiers from $999; enterprise and OEM licences are quoted per deployment. See OEM licensing.

Planning a family or security add-on?

Run 120 million categorized domains inside your own resolvers, with daily new-domain deltas and carrier-ready formats.

Talk to us