Streaming Classification API - Real-Time Domain Classification at Scale

Why Streaming Classification?

Traditional REST APIs introduce per-request overhead that becomes a bottleneck at scale. The Streaming Classification API uses persistent WebSocket connections to deliver sub-millisecond domain lookups at tens of thousands of requests per second, making filtering invisible to end users.

Real-Time Streaming

Persistent WebSocket connections stay open for hours or days, eliminating TCP handshake, TLS negotiation, and HTTP framing overhead on every lookup.

Low Latency Classification

Sub-millisecond wire latency with 99th percentile under 5ms. Designed for inline filtering where every millisecond of delay impacts the browsing experience.

Continuous Updates

Push-based category updates delivered in real time. When a domain is reclassified, all connected clients receive instant notifications without polling.

100M+

Domains Classified

<10ms

End-to-End Latency

59

WebSocket Protocol and Connection Management

Production-grade connection handling for always-on classification

1

Connection Establishment

Connect to the streaming endpoint using a standard WebSocket upgrade request with your API key in the authorization header. The server responds with connection metadata including your rate limit, supported protocol version, and a connection ID for troubleshooting. Connections authenticate once during the handshake and remain authorized for the lifetime of the session, eliminating repeated credential validation overhead on every lookup.

2

Message Framing

Each classification request is a compact JSON frame containing the domain string and an optional client-assigned request ID. Response frames echo the request ID paired with the classification result, enabling your client to correlate responses with requests even when messages arrive out of order. Binary framing mode is available for maximum efficiency, encoding domains as length-prefixed UTF-8 strings and results as fixed-width category codes.

3

Connection Keepalive

The server sends periodic ping frames at 30-second intervals, and the client must respond with pong frames to prove liveness. Connections that miss three consecutive pongs are terminated server-side. Clients should implement automatic reconnection with exponential backoff starting at 100 milliseconds. The reconnection handshake includes the previous connection ID, allowing the server to resume session state and avoid reclassifying recently seen domains.

4

Multiplexed Channels

A single WebSocket connection supports multiplexed logical channels, each with independent flow control and priority. Assign DNS query classifications to a high-priority channel that gets guaranteed bandwidth, while background log enrichment flows through a best-effort channel. This prevents bulk background work from starving latency-sensitive inline filtering requests, all without the overhead of managing multiple physical connections.

Event-Driven Classification Architecture

Push-Based Category Updates

Beyond request-response classification, the streaming connection delivers real-time category update events. When our classification engine recategorizes a domain, all connected clients with that domain in their recent lookup cache receive a push notification with the updated category. This ensures your filtering decisions reflect the latest intelligence without polling or cache invalidation delays. A domain that was categorized as "Business" yesterday but was compromised and reclassified as "unsafe content" today triggers an immediate push to all active connections. This real-time reclassification capability is also used to detect web signals of business owner retirement based on shifting domain activity patterns.

content risk Alert Stream

Subscribe to the content risk alert channel to receive real-time notifications about newly identified restricted content campaigns, unsafe content distribution networks, and command-and-control infrastructure. These alerts arrive as structured events containing the content risk type, affected domains, accuracy level, and recommended action. Your filtering infrastructure can ingest these alerts to proactively block emerging content risks before users encounter them, closing the window between content risk discovery and policy enforcement.

Connection-Level Analytics

The streaming API emits periodic analytics frames summarizing your connection's activity: lookups per second, cache hit ratio, category distribution of recent queries, and latency percentiles. These metrics integrate directly with monitoring systems like Prometheus, Datadog, or Grafana, giving network operations teams real-time visibility into classification performance without instrumenting their own client code.

Infrastructure Integration Points

Deploy streaming classification where it matters most in your network stack

DNS Resolver Integration

Embed streaming classification into recursive DNS resolvers to filter at the network's most fundamental layer. When a client queries a domain, the resolver classifies it over the persistent WebSocket before returning the DNS response. Blocked domains receive NXDOMAIN or a redirect to a block page. This approach requires no client software, works with every device on the network, and adds less than 2 milliseconds to DNS resolution time.

Forward Proxy Integration

Integrate with Squid, Apache Traffic Server, or custom proxy chains using our streaming client library. The proxy intercepts HTTP CONNECT requests and classifies the target domain before establishing the upstream tunnel. For HTTP traffic, the proxy inspects the Host header. Classification results are cached locally with a configurable TTL, so repeat requests to the same domain during a browsing session bypass the WebSocket entirely.

Next-Gen Firewall Integration

Feed streaming classification into firewall policy engines through syslog, STIX/TAXII, or custom connectors. Palo Alto, Fortinet, and Check Point firewalls can consume our category feeds to enrich their application-layer filtering rules. The streaming API provides the freshest category data possible, updating firewall policies within seconds of a domain reclassification rather than waiting for scheduled feed updates.

Wireless Controller Integration

Enterprise wireless controllers from Cisco, Aruba, and Ruckus can leverage streaming classification for per-SSID content filtering. Guest networks, BYOD networks, and corporate networks each receive tailored filtering through the same streaming connection differentiated by channel. Captive portal redirects for blocked content integrate with the controller's existing guest access framework.

Performance Characteristics and Scaling

Engineered for the most demanding network environments

Sub-Millisecond Latency

Median classification latency is 0.8 milliseconds measured server-side. With client-side caching of recently classified domains, effective latency drops further as cache hit rates of 60 to 80 percent are typical for enterprise traffic. The 99th percentile latency remains under 5 milliseconds even during peak load.

Horizontal Scaling

Open multiple WebSocket connections to different endpoint addresses for horizontal throughput scaling. Each connection sustains 10,000 lookups per second independently. A pool of 10 connections delivers 100,000 classifications per second, sufficient for the largest enterprise networks and ISP deployments. Load balancing across connections is handled by your client.

Local Cache Tiering

Our client libraries implement a two-tier local cache: an in-memory LRU cache for the hottest domains and an on-disk SQLite cache for the long tail. Cache entries include TTL metadata from the server, ensuring stale entries expire automatically. Cache warming on connection startup pre-fetches classifications for your most queried domains based on historical access patterns.

Edge PoP Network

Streaming endpoints are available in 12 global points of presence. Connections are routed to the nearest PoP using anycast DNS, minimizing network latency. If a PoP becomes unavailable, connections automatically fail over to the next nearest location. Each PoP maintains a full replica of the 100 million domain classification database for local query resolution.

Client Libraries and SDK Support

Production-ready libraries for every major platform

C/C++ Library

Zero-allocation streaming client for embedding in DNS resolvers, firewalls, and network appliances written in C. The library compiles to a single shared object with no external dependencies. Async I/O uses epoll on Linux and kqueue on BSD, matching the event model of high-performance network applications.

Go Module

Goroutine-safe streaming client with built-in connection pooling and graceful reconnection. The module exposes a simple Classify(domain) function that abstracts all WebSocket management. Fully compatible with CoreDNS plugin architecture for drop-in DNS filtering capability with minimal configuration.

Python asyncio Client

Async Python client built on the websockets library for integration with asyncio-based proxy servers and security tools. Supports both await-based single lookups and async generator patterns for processing domain streams. Compatible with popular frameworks including FastAPI, aiohttp, and Twisted.

Designing for Resilience

Production streaming deployments must handle connection interruptions gracefully. Our client libraries implement a circuit breaker pattern: if the WebSocket connection fails, the library falls back to REST API lookups for a configurable cooldown period before attempting to reestablish the streaming connection. During fallback, a local cache serves recently classified domains with zero latency, and only genuinely new domains incur REST overhead. Organizations that leverage classification data as one of their tools for finding owner-ready-to-sell businesses benefit from this resilient architecture.

For mission-critical deployments, run active-active streaming connections to two separate PoPs simultaneously. Both connections receive the same classifications, and your application selects whichever responds first. This eliminates single points of failure in the classification path and ensures that even a full PoP outage causes no interruption to filtering service. The dual-connection overhead is minimal since idle WebSocket connections consume negligible bandwidth.

Streaming Classification in Action

Classify streaming and media domains in real time. Connect via WebSocket and push domains for instant categorization of live content platforms.

import asyncio
import websockets
import json

async def classify_streams():
    uri = "wss://api.webfilteringdatabase.com/v1/stream"
    headers = {"Authorization": "Bearer YOUR_API_KEY"}

    async with websockets.connect(uri, extra_headers=headers) as ws:
        # Streaming & media domains to classify
        domains = ["twitch.tv", "tiktok.com", "spotify.com", "piracy-stream.xyz"]

        for domain in domains:
            await ws.send(json.dumps({"domain": domain}))
            result = json.loads(await ws.recv())
            print(f"{domain} -> {result['category']} (confidence: {result['confidence']})")

asyncio.run(classify_streams())
# twitch.tv -> Streaming Media (confidence: 0.98)
# tiktok.com -> Social Media (confidence: 0.99)
# spotify.com -> Streaming Media (confidence: 0.97)
# piracy-stream.xyz -> Piracy/Illegal (confidence: 0.95)

const WebSocket = require('ws');

const ws = new WebSocket('wss://api.webfilteringdatabase.com/v1/stream', {
    headers: { 'Authorization': 'Bearer YOUR_API_KEY' }
});

const domains = ['twitch.tv', 'tiktok.com', 'spotify.com', 'piracy-stream.xyz'];
let idx = 0;

ws.on('open', () => {
    // Send first domain when connection opens
    ws.send(JSON.stringify({ domain: domains[idx] }));
});

ws.on('message', (data) => {
    const result = JSON.parse(data);
    console.log(`${domains[idx]} -> ${result.category}`);
    idx++;
    if (idx < domains.length) {
        ws.send(JSON.stringify({ domain: domains[idx] }));
    } else {
        ws.close();
    }
});

# Connect to the streaming classification WebSocket
# Note: cURL supports WebSocket via --ws flag (v7.86+)

# Single domain lookup via REST fallback:
curl -X POST https://api.webfilteringdatabase.com/v1/classify \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"domains": ["twitch.tv", "tiktok.com", "spotify.com", "piracy-stream.xyz"]}'

# Response:
# {
#   "results": [
#     {"domain": "twitch.tv", "category": "Streaming Media", "confidence": 0.98},
#     {"domain": "tiktok.com", "category": "Social Media", "confidence": 0.99},
#     {"domain": "spotify.com", "category": "Streaming Media", "confidence": 0.97},
#     {"domain": "piracy-stream.xyz", "category": "Piracy/Illegal", "confidence": 0.95}
#   ]
# }

<?php
use WebSocket\Client;

// Streaming classification via WebSocket
$client = new Client("wss://api.webfilteringdatabase.com/v1/stream", [
    'headers' => ['Authorization' => 'Bearer YOUR_API_KEY']
]);

$domains = ['twitch.tv', 'tiktok.com', 'spotify.com', 'piracy-stream.xyz'];

foreach ($domains as $domain) {
    $client->send(json_encode(['domain' => $domain]));
    $response = json_decode($client->receive(), true);

    echo "{$domain} -> {$response['category']}";
    // twitch.tv -> Streaming Media
    // tiktok.com -> Social Media
    // spotify.com -> Streaming Media
    // piracy-stream.xyz -> Piracy/Illegal
}

$client->close();
?>

Ready for Real-Time Streaming Classification?

Classify domains at wire speed with persistent WebSocket connections backed by our 100 million domain database.

Check Domain See Pricing Download Sample