webfilteringdatabase.com
Home Find Your Solution
Features
Domain Categorization API Real-Time Classification 59 Filtering Categories Offline Database (100M) ML Classification Content Classification
Industries
K-12 Schools Corporate Healthcare Government ISPs
Tools
Domain Lookup Bulk Categorization Category Explorer
Resources
Pricing API Documentation Login / Sign Up
Knowledge Base

ISP Content Filtering

Carrier-Grade Domain Categorization for Clean-Pipe Services, Subscriber Management, and Regulatory Compliance Across Millions of Concurrent Users

Carrier-Grade Content Filtering

ISPs operate at a scale where every millisecond matters. With tens of billions of DNS queries per day across millions of subscribers, the web filtering database delivers sub-millisecond categorization through in-memory local mirrors and 15-minute incremental delta updates.

Subscriber-Scale Filtering

Handle millions of concurrent subscribers with in-memory database mirrors that eliminate external API latency at carrier-grade throughput.

DNS Integration

Deploy alongside existing DNS infrastructure with BGP-based traffic steering, anycast addressing, and seamless failover across filtering nodes.

Regulatory Compliance

Meet obligations under the UK Online Safety Act, Australia eSafety framework, and EU Digital Services Act with auditable filtering and transparency controls.

100M+
Domains Categorized
59
Content Categories
<10ms
Lookup Latency
Millions
Subscribers Served

Clean-Pipe Services & Revenue Opportunities

Transform web filtering from a cost center into a revenue-generating subscriber service

Clean-pipe services represent one of the most compelling revenue opportunities for ISPs. By offering tiered content filtering as an add-on subscription, ISPs can generate incremental ARPU (Average Revenue Per User) while simultaneously reducing support costs from unsafe content infections and customer complaints about inappropriate content, particularly among baby boomer-owned businesses and older subscribers who face heightened online safety risks.

A typical clean-pipe offering includes three tiers. The base tier, often included free with the connection, blocks unsafe content, and command-and-control domains -- protecting the ISP's network and reducing support burden. The family tier adds parental control categories including adult content, gambling, violence, and social media restrictions. The premium tier provides full category control through a subscriber portal, allowing customization of all 59 content categories with time-based scheduling and per-device policies.

The web filtering database makes these tiers possible through its granular 59-category taxonomy. Each subscriber's selected tier maps to a set of blocked categories. The filtering infrastructure checks each DNS query against the subscriber's active policy, returning the appropriate response. Block pages can be white-labeled with the ISP's branding and include upsell prompts for higher tiers.

Revenue Generation

Premium filtering tiers add $2-5/month ARPU with minimal infrastructure cost

Reduced Support Costs

unsafe content blocking reduces infection-related support calls by up to 40%

Subscriber Retention

Family safety features reduce churn by giving parents a reason to stay

Subscriber Management & Parental Controls

Per-subscriber policy management at scale with self-service portals

Per-Subscriber Policy Architecture

ISP-scale filtering requires the ability to maintain individual policies for millions of subscribers simultaneously. Each household may have different filtering preferences, and within a household, different devices may require different policies. A teenager's phone should have stricter filtering than a parent's laptop.

RADIUS/DHCP Integration Identify subscribers by RADIUS session or DHCP lease. Map each subscriber to their filtering policy without requiring endpoint software. Works with any CPE device.
Per-Device Profiles Subscribers create device profiles (child, teen, adult, guest) in the self-service portal. Device MAC addresses or hostnames map to profiles. Policies apply automatically when devices connect.
Time-Based Scheduling Parents set bedtime rules that restrict internet access or limit allowed categories after hours. School-night schedules can differ from weekend schedules. Schedules apply per-device.
Real-Time Override Subscribers can temporarily allow a blocked domain through a PIN-protected bypass mechanism. Overrides are logged and time-limited, automatically reverting after the configured period.

Self-Service Subscriber Portal

A white-labeled subscriber portal reduces support costs by enabling customers to manage their own filtering settings. The portal presents the 59 content categories in user-friendly groups such as "Adult Content," "Social Media," "Streaming," and "web-based risks." Subscribers toggle categories on or off, set schedules, manage device profiles, and view activity reports.

The portal integrates with the ISP's existing subscriber management system through standard APIs. Changes made in the portal propagate to the filtering infrastructure within seconds, providing immediate feedback. For ISPs using BSS/OSS platforms such as Amdocs, CSG, or Netcracker, the filtering database provides pre-built integration adapters.

Activity reports show subscribers which domains were blocked and why, building trust and reducing support calls from subscribers who do not understand why a site was blocked. Reports are anonymized at the household level to comply with privacy regulations while still providing useful insights for parents monitoring their children's browsing patterns.

ISP Deployment Architecture

Carrier-grade technical considerations for reliable, scalable content filtering

1

BGP-Based Traffic Steering

Use BGP announcements to redirect DNS traffic from subscriber-facing routers to the filtering DNS cluster. Anycast addressing ensures queries are routed to the nearest filtering node, minimizing latency. BGP health checks automatically withdraw routes from failed nodes, providing seamless failover without subscriber impact. This approach avoids inline deployment and works with any network topology.

2

CGNAT Subscriber Identification

Carrier-Grade NAT (CGNAT) complicates subscriber identification because multiple subscribers share a single public IP. The filtering system integrates with the CGNAT platform's session logging to map internal subscriber addresses to filtering policies. Alternative approaches include using DHCP option 82 (relay agent information), subscriber VLAN tags, or PPPoE session identifiers to maintain per-subscriber policy accuracy behind CGNAT.

3

Horizontal Scaling Architecture

The filtering cluster scales horizontally by adding DNS resolver nodes behind the anycast address. Each node maintains a local copy of the 90-million-domain database in memory, enabling independent operation without a central bottleneck. Load balancing happens automatically through anycast routing. Typical deployments start with four nodes for redundancy and scale to dozens of nodes for large ISPs, supporting millions of queries per second.

4

High Availability & Disaster Recovery

ISP filtering infrastructure must achieve 99.999% availability because DNS failures affect all subscriber internet access. Deploy filtering nodes in multiple data centers with independent power and network paths. Implement automatic failover: if all filtering nodes become unreachable, configure fallback to unfiltered resolution rather than blocking all DNS. Maintain offline database snapshots that can bootstrap new nodes within minutes.

Performance Benchmarks

Each filtering node handles 200,000+ queries per second with p99 latency under 1ms using in-memory database lookups. A four-node cluster provides 800K+ QPS with N+1 redundancy, sufficient for an ISP serving 2-3 million subscribers.

Memory Requirements

The full 90-million-domain database requires approximately 8GB of RAM when loaded into an optimized trie structure. Combined with resolver cache and operating system overhead, plan for 32GB per filtering node to ensure headroom for growth.

Update Propagation

Incremental database updates (IXFR) propagate to all nodes within 15 minutes. Emergency content risk updates can be pushed in under 5 minutes through an out-of-band notification channel. Nodes apply updates atomically without service interruption.

ISP Regulatory Obligations

ISPs face unique regulatory obligations around content filtering that differ significantly from enterprise requirements. These obligations vary by jurisdiction but share common themes: protecting minors, blocking illegal content, and providing transparency to subscribers. Understanding how to identify businesses with aging owners can also help ISPs tailor their enterprise filtering solutions to this growing demographic.

UK Online Safety Act: Requires ISPs to implement measures to prevent children from accessing pornographic content online. The Act mandates age verification mechanisms and content filtering as part of a layered approach to child safety. ISPs that fail to comply face significant fines from Ofcom.

Australia eSafety Framework: The eSafety Commissioner can issue blocking notices requiring ISPs to prevent access to specific domains hosting illegal content. The web filtering database supports rapid integration of these blocking notices into the filtering infrastructure.

EU Digital Services Act: While not mandating filtering directly, the DSA requires ISPs to act on valid takedown notices and implement measures against the dissemination of illegal content. Proactive filtering demonstrates good-faith compliance and reduces the volume of reactive takedown requests.

US CIPA / E-Rate: ISPs that serve American schools and libraries must be able to support CIPA and E-Rate compliant filtering for school districts, since those customers are legally required to filter internet access as a condition of their federal funding.

Net Neutrality Considerations: Content filtering must be implemented transparently and with subscriber consent. Subscribers must be informed about what categories are filtered, have the ability to opt out (where legally permitted), and filtering must not discriminate based on the source of content or favor the ISP's own services.

API Integration for ISP Subscriber Filtering

Integrate carrier-grade domain categorization into your subscriber management pipeline

ISP Subscriber Content Filter

Classify subscriber DNS requests against 59 content categories and enforce per-subscriber clean-pipe policies in real time.

import requests

API_URL = "https://webfilteringdatabase.com/api/moderate.php"
API_KEY = "YOUR_API_KEY"

# Subscriber tier policies
TIERS = {
    "base": {"Malware", "Phishing", "Botnet"},
    "family": {"Malware", "Phishing", "Adult Content",
               "Gambling", "Violence", "Drugs"},
    "premium": {"Malware", "Phishing", "Adult Content",
                "Gambling", "Violence", "Drugs",
                "Social Media", "Streaming"}
}

def filter_subscriber_request(domain, tier):
    result = requests.post(API_URL, json={
        "api_key": API_KEY,
        "query": domain
    }).json()

    categories = set(result.get("categories", []))
    blocked = TIERS.get(tier, TIERS["base"])
    violations = categories & blocked

    if violations:
        return "BLOCK", f"Policy: {tier} | {', '.join(violations)}"
    return "ALLOW", result.get("primary_category", "Uncategorized")

# Simulate subscriber DNS requests
domains = [
    "streaming-service.com",
    "adult-site.xxx",
    "news.com",
    "social-media.com"
]
for d in domains:
    action, reason = filter_subscriber_request(d, "family")
    print(f"{action}: {d} — {reason}")
const API_URL = 'https://webfilteringdatabase.com/api/moderate.php';
const API_KEY = 'YOUR_API_KEY';

const TIER_POLICIES = {
    base: new Set(['Malware', 'Phishing', 'Botnet']),
    family: new Set(['Malware', 'Phishing', 'Adult Content',
        'Gambling', 'Violence', 'Drugs']),
    premium: new Set(['Malware', 'Phishing', 'Adult Content',
        'Gambling', 'Violence', 'Social Media', 'Streaming'])
};

async function filterSubscriberRequest(domain, tier) {
    const res = await fetch(API_URL, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ api_key: API_KEY, query: domain })
    });
    const data = await res.json();
    const blocked = TIER_POLICIES[tier] || TIER_POLICIES.base;
    const violations = (data.categories || [])
        .filter(c => blocked.has(c));

    if (violations.length) {
        return { action: 'BLOCK', reason: violations.join(', ') };
    }
    return { action: 'ALLOW', category: data.primary_category };
}

// Check subscriber requests
['streaming-service.com', 'adult-site.xxx',
 'news.com', 'social-media.com']
    .forEach(async d => {
        const r = await filterSubscriberRequest(d, 'family');
        console.log(`${r.action}: ${d}`);
    });
# Check a streaming domain for subscriber filtering
curl -X POST "https://webfilteringdatabase.com/api/moderate.php" \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_API_KEY",
    "query": "streaming-service.com"
  }'

# Response: {"primary_category":"Streaming","risk_level":"low"}

# Check an adult domain
curl -X POST "https://webfilteringdatabase.com/api/moderate.php" \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_API_KEY",
    "query": "adult-site.xxx"
  }'

# Response: {"primary_category":"Adult Content","risk_level":"high"}

# Check a news domain
curl -X POST "https://webfilteringdatabase.com/api/moderate.php" \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_API_KEY",
    "query": "news.com"
  }'

# Response: {"primary_category":"News","risk_level":"low"}

# Check a social media domain
curl -X POST "https://webfilteringdatabase.com/api/moderate.php" \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "YOUR_API_KEY",
    "query": "social-media.com"
  }'

# Response: {"primary_category":"Social Media","risk_level":"low"}
<?php
$apiUrl = 'https://webfilteringdatabase.com/api/moderate.php';
$apiKey = 'YOUR_API_KEY';

$tiers = [
    'base'    => ['Malware', 'Phishing', 'Botnet'],
    'family'  => ['Malware', 'Phishing', 'Adult Content',
                  'Gambling', 'Violence', 'Drugs'],
    'premium' => ['Malware', 'Phishing', 'Adult Content',
                  'Gambling', 'Violence', 'Social Media']
];

function filterSubscriberRequest($domain, $tier, $apiUrl, $apiKey, $tiers) {
    $ch = curl_init($apiUrl);
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode([
        'api_key' => $apiKey,
        'query'   => $domain
    ]));
    curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/json']);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    $result = json_decode(curl_exec($ch), true);
    curl_close($ch);

    $blocked = $tiers[$tier] ?? $tiers['base'];
    $violations = array_intersect($result['categories'] ?? [], $blocked);

    if (!empty($violations)) {
        return ['BLOCK', implode(', ', $violations)];
    }
    return ['ALLOW', $result['primary_category'] ?? 'Unknown'];
}

// ISP subscriber DNS request filtering
$domains = ['streaming-service.com', 'adult-site.xxx', 'news.com', 'social-media.com'];
foreach ($domains as $d) {
    [$action, $reason] = filterSubscriberRequest($d, 'family', $apiUrl, $apiKey, $tiers);
    echo "$action: $d — $reason\n";
}
?>

Deploy Carrier-Grade Content Filtering

Partner with us to bring 100 million categorized domains to your subscriber base. Customizable clean-pipe services, white-label portals, and ISP-grade SLAs.