URL Category Data for SASE Platforms
A SASE platform makes millions of destination decisions an hour across its gateway, DNS layer, firewall rules and cloud app controls. Every one of those decisions is better when it knows what kind of site sits behind a domain.
This guide shows where URL category data plugs into a SASE stack, which categories carry most of the policy weight, and how to deliver the data to points of presence without adding latency.
Where category data plugs into a SASE stack
SASE joins network and security services that used to be separate boxes. Each service asks the same question about a destination, so one category store should answer all of them.
The main consumer. Allow, block, warn, isolate or inspect decisions by category on every web request, after TLS inspection where policy allows it.
Answers or blocks at resolution time, before a connection opens. It is fast and protects devices where full proxying is not possible.
Uses categories as objects in rules, often matched on the TLS server name. This lets admins write one rule for a whole class of sites instead of IP lists.
Discovery and control of SaaS use, including unsanctioned apps. Categories group the long tail of apps that no app catalog covers by name.
Mostly about internal apps, but category data helps decide what happens to internet traffic from the same agent. See the URL category data for ZTNA guide.
Turns raw destination logs into readable usage reports by category, per site, team and tenant. This is often the feature customers look at first.
Which categories carry most SASE policy
Enterprise customers use a fairly small set of categories for most rules, and the rest feed reporting and acceptable use. Here is how the groups usually split across a SASE service.
| Category | Typical SASE action | Why it matters | Weight |
|---|---|---|---|
| Malware / Phishing / Suspicious | Block at DNS and gateway | The baseline security rule every tenant turns on first | Core |
| VPN / Proxy / Anonymizers | Block or alert | Proxies and anonymizers route around the SASE service itself | Core |
| Remote Access Tools | Allow only sanctioned tools | Common in support scams and in hands-on-keyboard intrusions | Core |
| AI / LLM Tools | Allow sanctioned, warn or isolate others, inspect uploads | Shadow AI is now a standard board-level question | Core |
| Cloud Storage / File Hosting, File Sharing/P2P | Allow corporate tenant, restrict uploads elsewhere | Main route for data leaving the company | Core |
| Parked / For-Sale Domains, URL Shorteners | Warn or block | Low business value, often used to hide the real destination | Often |
| Adult, Gambling, Weapons, Illegal Drugs, Hate/Discrimination | Block for acceptable use | HR and legal policy, plus liability on shared devices | Often |
| Online Banking, Health & Medicine, Government & Legal | Exclude from TLS inspection | Privacy rules and works council agreements limit decryption | Often |
| Streaming Media, Social Networking, Gaming | Bandwidth rules, schedules | Traffic steering and branch bandwidth control | Situational |
| News/Media, Shopping, Travel and other content categories | Report only | Usage reporting and audit trails | Situational |
Delivery: files, lookup API and the PoP working set
A SASE service runs in many points of presence. The data needs to be close to each enforcement node without making every PoP carry the same load.
In-memory working set
Load the most visited domains into memory on each enforcement node. A local lookup has no network round trip, so it fits the per-request budget.
- Built from the full file
- Sized to the node's memory
- Rebuilt after each delta
Full regional store
Keep the complete database in a regional key-value store. Nodes query it on a working-set miss, which stays inside your own network.
- CSV, Parquet or JSON import
- Daily delta files applied once
- Fans out to all nodes
Lookup API
Use the API for domains neither tier holds and for support tooling. Single and batch lookups of up to 1,000 domains per request are available.
- Included with licences
- Async path, never inline
- Results written back to cache
Latency budget on the request path
Customers judge a SASE service by how fast pages load through it. The category lookup must never be the slow part of the chain.
Extract the host
Take the host from the DNS query, the TLS server name or the HTTP request. Normalize case and strip trailing dots.
Exact, then parent
Look up the full host first, then walk up to the registered domain. Subdomain-specific labels win over parent labels.
Decide locally
Apply tenant policy using the labels found in memory. No external call happens before the decision.
Handle misses async
On a miss, apply the tenant's default for unknown domains and queue the domain for the regional store or API.
| Path | Speed need | Source | Notes |
|---|---|---|---|
| DNS answer | Inline, strict | Node memory only | Resolvers cannot wait on a remote call |
| Gateway request decision | Inline | Node memory, regional store on miss | Cache the regional answer on the node |
| Firewall rule match | Inline | Node memory | Match on server name before the session is set up |
| Cloud app discovery | Minutes | Regional store, batch API | Runs over logs, not live traffic |
| Tenant reports | Hours | Full file | Re-label old logs with current categories if needed |
Policy examples SASE customers expect
These rule patterns show up in nearly every enterprise tenant. Shipping them as defaults shortens onboarding and makes the category data visible from day one.
Shadow AI control
IF category = "AI / LLM Tools"
AND app NOT IN sanctioned_ai
THEN warn + isolate, block file uploadLets staff read answers from unsanctioned assistants while keeping company files out of them.
Anonymizer block
IF category = "VPN / Proxy / Anonymizers"
THEN block, alert security teamStops users from tunnelling around the service. Pair it with an app-level rule for VPN clients on the device.
Inspection bypass for privacy
IF category IN ("Online Banking",
"Health & Medicine")
THEN do not decrypt, log category onlyKeeps personal banking and medical sessions private while still recording that a visit took place.
Remote access allow list
IF category = "Remote Access Tools"
AND domain NOT IN it_approved
THEN blockAllows the support tool IT has chosen and blocks the dozens of others that attackers like to use.
Personal storage uploads
IF category = "Cloud Storage / File Hosting"
AND tenant != corporate_tenant
THEN allow download, block uploadNeeds instance awareness from the gateway, but the category tells it which sites to inspect closely.
Young and unknown domains
IF domain in recent new-domain deltas
OR category = unknown
THEN isolate or warnMany phishing pages live on domains a few days old. Isolation keeps users productive while risk is unclear.
Freshness: what changes and how fast
Threat-heavy categories change daily, content categories change slowly. The update plan should reflect that difference.
About 300,000 newly registered domains are categorized and shipped in the daily delta file.
Your pipeline applies the delta to regional stores and rebuilds node working sets.
New domains are checked again, because many sites only add content after launch.
A second recheck of the same new domains arrives in the deltas.
All other domains are refreshed. Merge or reload the full table on your own schedule.
Mapping categories to your own taxonomy
Most SASE vendors already have category names in their console and in customer policies. You keep those names and map our labels underneath.
One-to-one
Most of the 59 categories map directly to a category you already show. Store the mapping as a small table in your product.
Many-to-one
Where your console groups things more broadly, several of our labels roll into one of yours. Ranked labels let you keep detail for reports.
New groups
Some categories, such as AI / LLM Tools or Remote Access Tools, may be new to your console. Adding them is often the reason teams change data provider.
Multi-tenant concerns
SASE is a shared service. Category data is shared across tenants, while policy and logs stay separate.
- Category labels describe public websites, so one shared store serves every tenant.
- Tenant overrides live in a separate table and win over the shared label for that tenant only.
- Logs record the domain, labels returned and rule applied, so admins can answer "why was this blocked".
- A "report a category" button feeds disputes back to you, and you pass them on for review.
- Default actions for unknown domains are set per tenant, not globally.
- Regional stores keep lookups inside the region a tenant has chosen.
Integration mistakes that hurt SASE products
These problems show up in support tickets months after launch. Each one is cheap to avoid during design.
Remote lookups inline
Calling an external service before every decision adds delay and a new failure point. Keep inline decisions on local data and push misses to an async queue.
Different answers per service
If the DNS layer and the gateway load different file versions, a site can be blocked in one and allowed in the other. Version every build and roll it out to all services together.
Top label only
A file sharing site that also hosts adult content needs both labels checked by block rules. Match security and acceptable use rules against every label, not only the first.
Ignoring subdomains
Large platforms host very different content on different hosts. Look up the full host before the registered domain so specific labels are not lost.
Silent blocks
A blocked page without a reason creates tickets. Show the category on the block page and offer a request path that reaches the tenant admin.
No delta monitoring
A missed delta day leaves a gap in new-domain coverage. Alert when a delta is late or when its size is far from normal.
A rollout plan for a SASE vendor
Teams that switch or add a category source usually move in four stages. Running old and new data side by side first makes the switch safe for customers.
Shadow mode
Load the file next to your current source and log both answers. Measure where they differ on real tenant traffic.
Map and tune
Finalize the mapping to your console categories. Review the largest groups of differences with your policy team.
Reports first
Switch analytics and reports to the new labels. Customers see better reporting before any enforcement changes.
Enforce by region
Move enforcement one region at a time, watching block rates and support tickets after each step.
Evaluating a category source for SASE
Run these tests with your own traffic before you commit. They take days, not months, and show real fit.
| Test | How | Good result |
|---|---|---|
| Coverage | Run a week of anonymized destination domains from a pilot tenant through bulk domain lookup | Very few unknowns among frequently visited domains |
| New-domain coverage | Test domains registered in the last few days | Most already carry labels |
| Taxonomy fit | Map your existing console categories | No category left without a source |
| Load time | Build a node working set from the file | Fits your memory and rebuild window |
| Licence fit | Describe every place customers see a category | Covered in writing by the OEM licence |
SASE category data questions
Where does URL category data sit in a SASE platform?
In every component that makes a decision about a destination: the secure web gateway, the DNS security layer, firewall as a service rules, the cloud access controls and the reporting pipeline. One shared category store keeps those decisions consistent.
Should each point of presence hold the full database?
Most platforms hold a working set of the most visited domains in memory at every point of presence and keep the full file in a regional store. The lookup API covers rare misses, and the 20 million most visited domains cover 99.99% of web traffic and usage.
How fresh does SASE category data need to be?
New domains matter most because phishing and scam sites are usually young. About 300,000 newly registered domains are categorized every day as daily delta files, with rechecks after a week and a month, and all other domains are refreshed monthly.
How many categories are included?
59 web filtering categories plus the IAB Content Taxonomy version 3 with 703 categories, in one file. A domain can carry several categories, ranked by how much of the site each one describes.
Can we show our own category names to customers?
Yes. Most OEM customers map our categories to their own product taxonomy and display their own names, and the licence covers showing categories inside your service.
Does the data cover AI tools for shadow AI controls?
Yes. The AI / LLM Tools category is backed by 11,800+ AI-tool domains in 18 categories and 165 subcategories, updated daily, so policies can treat chat assistants, code assistants and image generators differently.
Can the data run in sovereign or air-gapped regions?
Yes. The database is delivered as files that run inside your own systems, so regional points of presence can operate without calling out to us.
Which licence does a SASE vendor need?
An OEM licence, because the data powers decisions and reports your customers see. Companies with several products can take a group-wide licence, and resale or publication of the raw data is not allowed under any licence.
Put one category engine behind your whole SASE stack
Tell us how many points of presence you run and which services need categories. We will propose a file and API setup and an OEM licence that fits.
Talk to us OEM licensing