URL Category Data for XDR Products
XDR collects telemetry from endpoints, networks, email, identity and cloud. Almost every one of those events contains a domain, and a bare domain tells an analyst very little.
This guide is for XDR product and detection engineering teams. It covers how URL categories enrich detections and investigations, which categories make good signals, and how to deliver the data into a high-volume pipeline.
Where category data plugs into an XDR pipeline
Categories are useful at every stage, from ingest to the final report. Most products add them at ingest so every later stage can rely on them.
Ingest
Extract domains from DNS, proxy, firewall, endpoint and email events. Add labels as fields.
Detect
Rules and models use the category as a feature, alone or with process and volume.
Correlate
Group events into incidents by category patterns across hosts and users.
Triage
Show labels next to every domain in the alert view. Analysts skip a lookup per domain.
Respond
Playbooks branch on category: isolate a host for anonymizer use, open a DLP case for exfiltration.
Hunt and report
Hunters filter by category; reports show exposure by category over time.
Categories that make strong detection signals
Some categories are rarely contacted by servers or by unusual processes, so contact is worth a closer look. The ATT&CK references are examples of related techniques.
| Category | Signal | Related technique examples | Pair with |
|---|---|---|---|
| Cloud Storage / File Hosting | Large uploads from servers, scripts or archive tools | Exfiltration to cloud storage (T1567.002) | Bytes out, process name, time of day |
| File Sharing/P2P | Any contact from a server, or tool downloads to endpoints | Exfiltration over web service (T1567), tool transfer | File writes after download |
| Remote Access Tools | Unapproved remote control software calling home | Remote access software (T1219) | New install events, user context |
| VPN / Proxy / Anonymizers | Traffic hidden from inspection | Proxy (T1090) | Process, destination port |
| Phishing, Malware, Suspicious | Clicks from email, downloads, callbacks | Phishing (T1566) | Email events, credential use after click |
| URL Shorteners | Short links in mail and chat that hide destinations | Phishing delivery | Resolved final host |
| Parked / For-Sale Domains | Repeated beacons to parked or very new domains | Web protocols for command and control (T1071.001) | Beacon timing, domain age |
| Developer Tools / Code Hosting | Scripts pulling code from public repositories | Web service abuse (T1102) | Parent process, user role |
| AI / LLM Tools | Unusual API use from servers or service accounts | Data exposure, insider risk | Volume, account type |
| Cryptomining | Mining pool contact from corporate hosts | Resource hijacking (T1496) | CPU use, process |
Detection examples built on categories
These detections combine a category with one or two other facts. The category narrows the haystack; the other facts make the alert precise.
Server uploads to personal storage
A server or domain controller sends a large volume to a storage or file sharing domain.
- Host role: server
- Category: Cloud Storage / File Hosting, File Sharing/P2P
- Bytes out above the host's normal level
New remote access tool after a phishing click
A user clicks a phishing link, and within an hour a remote access tool contacts its service from the same device.
- Event 1 category: Phishing
- Event 2 category: Remote Access Tools
- Same host, short time window
Script talking to an anonymizer
A scripting engine or unsigned binary connects to a proxy or anonymizer domain.
- Process: interpreter or unsigned
- Category: VPN / Proxy / Anonymizers
- Not a known corporate VPN
Beaconing to parked or brand-new domains
Regular small connections from one host to a domain that is parked or arrived in recent new-domain deltas.
- Category: Parked / For-Sale, or new and unlabelled
- Regular interval
- Few other hosts contact it
Code pulled from public repositories by office apps
A document process or office macro fetches content from code hosting.
- Parent: office application
- Category: Developer Tools / Code Hosting
- File written to disk
Mining pool contact
A corporate host contacts a cryptomining domain, often after a vulnerable service was exposed.
- Category: Cryptomining
- Sustained high CPU
- Host type: server or cloud workload
An investigation, with and without categories
Here is a short incident timeline as an analyst would see it. The category column is what turns a list of domains into a story.
Enrichment fields to add
Keep the schema small and stable. These fields cover detection, triage and hunting needs without bloating storage.
| Field | Content | Used by |
|---|---|---|
dest_categories | All labels for the host or domain, in rank order | Detections, filters, reports |
dest_primary_category | The top-ranked label | Dashboards and grouping |
dest_category_group | Your own grouping, such as security, exit route, business | Simple rules and UI badges |
dest_new_domain | Whether the domain arrived in recent new-domain deltas | Beaconing and phishing rules |
dest_label_source | Host-level label or parent-domain label | Analyst confidence |
label_version | Date of the data build used at enrichment | Audits and re-processing |
dest_iab_category | Optional IAB Content Taxonomy label for content detail | Insider risk and reporting |
Delivery into a high-volume pipeline
XDR platforms process very large event volumes. Category enrichment has to be a local, in-memory step, not a remote call per event.
Ingest enrichment service
Load the most visited domains into memory in each enrichment worker. The 20 million most visited domains cover 99.99% of web traffic and usage.
- Built from the licensed file
- Hot-swapped after each delta
- Host lookup, then parent
Full table in the data lake
Keep all 120 million domains as a table next to customer data, from Parquet or CSV. Use it for misses and query-time joins.
- Partition by registered domain
- Versioned by build date
- Joins for hunting over old data
Lookup API for gaps
Look up the few domains neither tier holds, asynchronously, and write answers back. Batch requests take up to 1,000 domains.
- Included with licences
- Domain only, never log lines
- Also useful in analyst tools
Freshness and point-in-time labels
Investigations look backwards, so the label at the time of the event matters. New attacker domains also need labels fast.
Daily deltas
About 300,000 newly registered domains are categorized every day. Apply the delta to enrichment workers and the lake table each day.
Rechecks
New domains are checked again after a week and after a month, because many sites add content after launch. Changes arrive in the deltas.
Monthly refresh
All other domains are refreshed monthly. Keep previous builds for a while so you can explain past detections.
Hunting queries analysts run with categories
Once labels are in the data, hunting becomes a matter of filters. These pseudo-queries show common starting points.
Servers talking to exit routes
events where host_role = "server"
and dest_categories has any of
("Cloud Storage / File Hosting","File Sharing/P2P")
summarize bytes_out by host, dest_domainFinds data leaving infrastructure that should never use personal storage.
Rare remote access tools
events where dest_categories has "Remote Access Tools"
and dest_domain not in approved_rmm
summarize hosts = dcount(host) by dest_domain
order by hosts ascTools seen on only one or two hosts deserve a closer look.
New domains contacted by many hosts
events where dest_new_domain = true
summarize hosts = dcount(host) by dest_domain
where hosts > 5A brand-new domain reached by several machines can mean a campaign or a supply chain issue.
Anonymizer use by account type
events where dest_categories has
"VPN / Proxy / Anonymizers"
summarize count() by account_type, processService accounts using anonymizers are a strong anomaly.
Response playbooks that branch on category
Automated response works best when the next step depends on what kind of destination was involved. Categories give playbooks a simple, reliable branch point.
| Trigger category | First automated step | Analyst step | Hand-off |
|---|---|---|---|
| Phishing | Pull the message from all mailboxes, reset the session token if the user submitted a form | Check sign-in logs for the account in the next hours | Identity team |
| Remote Access Tools, unapproved | Kill the process, block the domain for the host | Confirm with the user whether a support call took place | Help desk, endpoint team |
| Cloud Storage or File Sharing with large upload | Snapshot process and file activity, notify data owner | Identify which files left, using DLP and endpoint logs | DLP and insider risk team |
| VPN / Proxy / Anonymizers from a server | Isolate the host from the network | Look for persistence and lateral movement | Incident response |
| Cryptomining | Stop the process, block the pool domain | Find the exposed service used to get in | Infrastructure team |
| New, unlabelled domain with beaconing | Look the domain up asynchronously, raise priority if it stays unknown | Review process tree and timing | Threat intelligence team |
Correlation patterns across data sources
XDR is valuable because it joins sources. Categories are a shared language across those sources, so one label can link an email, a DNS query and an endpoint event.
Email to endpoint
A link category in an email event matches the DNS and process events that follow the click on the same host.
Identity to network
A risky sign-in followed by uploads to storage categories from the same account forms one incident, not two.
Cloud to endpoint
A cloud workload and a laptop that both contact the same new domain are linked, even when the domain has no threat record yet.
Mistakes to avoid
These are the common ways category enrichment goes wrong in detection platforms.
Alerting on category alone
A visit to a file sharing site is not an incident. Combine the category with process, host role, volume or sequence.
Overwriting old labels
Re-enriching history with today's labels hides what the detection saw. Keep the label and build date at ingest.
Remote calls per event
Calling any API inline per event will not scale and adds cost. Enrich from memory and resolve misses asynchronously.
Ignoring the host level
Shared platforms host very different content on different hosts. Enrich on the full host first, then the registered domain.
Licensing for XDR vendors
Categories shown in alerts, investigations and reports are an embedded use of the data, covered by an OEM licence.
OEM licence
For XDR, MDR and SIEM products you sell, where customers see categories in detections and reports.
Group-wide licence
For vendors with endpoint, network, email and XDR products under one company.
Not allowed
Resale or publication of the raw data, such as exposing the full domain table as a downloadable lookup in your console.
XDR category data questions
What does URL category data add to an XDR product?
Context. Every domain in endpoint, network, DNS, email and identity telemetry gets a readable label, so detections can use it, correlation can group events by it, and analysts see at a glance what kind of site was contacted.
Is category data a threat intelligence feed?
Not only. It labels every domain, good and bad, across 59 web filtering categories, including Malware / Phishing / Suspicious, while threat feeds list known bad indicators.
Should enrichment happen at ingest or at query time?
Both have a place. Ingest-time enrichment stores the label that was true when the event happened and makes detections fast, while query-time joins against the current file are useful for hunting over older data.
How large is the dataset to load?
The full database holds 120 million categorized domains, delivered as CSV, Parquet or JSON. The 20 million most visited domains cover 99.99% of web traffic and usage, so many products keep that working set in memory.
How fresh is the data?
About 300,000 newly registered domains are categorized every day and shipped as daily delta files, with rechecks after a week and after a month. All other domains are refreshed monthly.
Can categories help detect data exfiltration?
Yes. Large outbound transfers from servers or unusual processes to Cloud Storage / File Hosting or File Sharing/P2P domains are a strong signal, and pairing the category with process and volume data keeps false positives down.
Do customer logs need to leave the XDR platform?
No. The licensed database runs inside your own systems, and the optional lookup API only needs a domain, never full URLs or log lines.
What licence does an XDR vendor need?
An OEM licence, since categories appear in detections, alerts and reports your customers see. Group-wide licences cover several products, and resale or publication of the raw data is not allowed.
Can categories reduce alert volume in an XDR product?
Yes, rules that ignore normal business categories and focus on exit routes, anonymizers, remote access tools and very new domains produce fewer, better alerts. Categories also let analysts filter ordinary browsing out of an incident timeline.
Does the data include a label for very new domains?
Newly registered domains are categorized daily, about 300,000 a day, and rechecked after a week and a month. Products can also mark domains that arrived in recent deltas as new, which is a useful signal on its own.
Give every domain in your telemetry a label
Tell us your event volume, data lake and detection stack. We will suggest how to load the files, size the in-memory set and license the data for your XDR product.
Talk to us OEM licensingMITRE ATT&CK is a registered trademark of The MITRE Corporation, used here only to identify the knowledge base referenced. Web Filtering Database is not affiliated with, sponsored or endorsed by The MITRE Corporation.