URL Category API for SaaS Products
Links flow through almost every SaaS product: chat messages, tickets, form fields, uploaded documents, ad creatives, customer websites. Knowing what each link points to opens up features your customers will pay for.
This page is for SaaS product and engineering teams deciding how to add URL categories: which features they enable, which categories matter, and when to use an API, a licensed database, or both.
Features URL categories unlock in a SaaS product
Most teams arrive with one feature in mind and discover three more once the data is in place. These are the features SaaS products build most often on URL categories.
Link safety in collaboration tools
Warn or block when a message, comment or ticket contains a link to phishing, malware or adult sites. Admins get a policy setting instead of a moderation queue.
Key: Malware / Phishing / Suspicious, AdultSign-up and KYC screening
Check the website a new customer enters at sign-up. Flag parked domains, gambling or adult sites before they get access to a payment or marketing product.
Key: Parked / For-Sale Domains, Gambling, AdultAnalytics enrichment
Turn referrer and destination domains into readable groups, such as "traffic from news sites" or "outbound clicks to shopping".
Key: all 59 categories, IAB taxonomy for detailAd and content review
Check landing pages in ad creatives, affiliate links and sponsored posts against category rules before they go live.
Key: Gambling, Alcohol, Tobacco, Weapons, Illegal DrugsWorkplace policy features
Browser-based workspace and device products add category policies for their business customers, including AI tool controls.
Key: AI / LLM Tools, Cloud Storage / File Hosting, VPN / ProxyMarketplace and UGC moderation
Listings, profiles and reviews often carry outbound links. Categories help moderation teams sort what to remove, what to review and what to allow.
Key: Adult, Dating, Hate/Discrimination, MarketplacesWhich categories matter for SaaS use cases
Every SaaS feature uses a subset of the 59 categories. The matrix shows where each group of categories usually matters most. See the full list on the category list.
| Category group | Link safety | Sign-up screening | Analytics | Ad review | Workplace policy |
|---|---|---|---|---|---|
| Malware / Phishing / Suspicious | |||||
| Adult, Lingerie/Swimsuits/Intimate Apparel | |||||
| Gambling, Alcohol, Tobacco, Weapons, Illegal Drugs | |||||
| Parked / For-Sale Domains | |||||
| URL Shorteners | |||||
| AI / LLM Tools, Cloud Storage, File Sharing/P2P | |||||
| News, Shopping, Business, Travel and other content categories |
API, licensed database, or both
The right answer changes as your feature grows. Many SaaS teams start on the API and add the licensed database later, keeping the API for domains the file does not hold.
Lookup API
Call the API for each new domain and cache answers.
- Days to first feature
- No data pipeline to run
- Single and batch endpoints
- Cost grows with volume
- Each lookup is a network call
Database plus API
Load the licensed file into your own store, and send misses to the API.
- Lookups in your own data store
- Predictable cost at high volume
- Customer URLs stay on your platform
- Daily delta files keep it fresh
- API covers the long tail
Database only
Run fully inside your own systems, including air-gapped regions.
- No outbound calls at all
- Fits data residency rules
- Same files, same updates
- Unknown domains need a default rule
- Most suited to security products
A reference architecture for SaaS lookups
Whatever stage you are at, the same five parts appear. Building them in from the start makes the later move from API to database a configuration change, not a rewrite.
Normalize before you look up
Strip scheme, path, query and fragment. Look up the full host, then fall back to the registered domain. This alone cuts lookups sharply.
Share the cache across tenants
Categories describe public websites, not customer data. One shared cache means a domain seen by any tenant is resolved for all.
Keep decisions per tenant
The category is shared, the policy is not. Store tenant rules separately and apply them after the lookup.
Batch the background work
For analytics and log enrichment, queue domains and use batch lookups of up to 1,000 domains. Keep single lookups for interactive paths.
Latency, volume and cost planning
SaaS features have different speed needs. A chat link warning must feel instant; a weekly analytics report can wait. Plan each path on its own.
| Path | Speed need | Recommended source | Why |
|---|---|---|---|
| Link preview or warning in chat | Instant | Local file or warm cache | Users notice any delay before a message renders |
| Sign-up website check | Seconds | API or local file | One lookup per sign-up; accuracy matters more than speed |
| Ad or listing review queue | Minutes | Batch API or local file | Many domains at once, perfect for batching |
| Analytics enrichment | Hours | Local file, batch for misses | Very high volume, mostly repeat domains |
| Workplace policy enforcement | Instant | Local file at the edge | Decisions sit on the browsing path |
Multi-tenant design and customer-facing settings
SaaS buyers expect control. These patterns let each customer shape the feature without touching the shared data.
Category toggles per tenant
Expose a short list of switches mapped to categories, such as "Block links to adult sites". Keep the mapping server-side so you can refine it later.
Tenant allow and block lists
Customers always know a domain you do not. Let them add exceptions that override the category for their account only.
Audit trail of decisions
Log the domain, the categories returned and the rule applied. Admins need to answer "why was this link blocked?" without contacting support.
Report a wrong category
Give admins a way to flag a category they disagree with. Apply their override at once, and pass the report to your data provider for review.
Update cadence: regular refreshes
Phishing links and fresh scam storefronts often live on domains registered days earlier. Freshness matters most for safety features, less for analytics.
| Update | What arrives | What to do in your product |
|---|---|---|
| Daily delta files | About 300,000 newly registered domains, plus label changes | Apply to your store daily and expire matching cache entries |
| Recheck after a week | Updated labels for new domains that added content | Arrives in the deltas; re-evaluate pending review items |
| Recheck after a month | Second pass on new domains | Arrives in the deltas; no extra work |
| Monthly refresh | All other domains refreshed | Reload or merge the full file on a schedule |
| API answers | Always current at request time | Set cache lifetimes in days, shorter for safety paths |
Integration steps
A typical SaaS integration runs through these steps. Steps 1 to 5 work on the API alone; steps 6 to 8 add the licensed database.
Pick the first feature
Choose one feature and list the categories it needs. A narrow start gets you to production faster and gives you real volume numbers.
Build the normalizer
Write one function that turns any URL your product sees into a host and a registered domain. Use it everywhere.
Add the cache
Put a shared cache in front of all lookups with a set lifetime. Log hit rate from day one.
Call the API
Use single lookups on interactive paths and batch lookups for queues. Handle quota responses with backoff.
Ship tenant policies
Expose category toggles, exceptions and an audit log in the admin console.
Load the licensed file
When volume justifies it, import the CSV, Parquet or JSON file into your own store, indexed by domain.
Run the delta job
Apply daily delta files, expire cache entries for changed domains, and alert if a day is missed.
Switch the resolver order
Resolve from the local store first and the API only on a miss. Your feature code does not change.
Privacy and security questions your customers will ask
Enterprise buyers review any feature that touches their URLs. Prepare answers to these questions before the first security questionnaire arrives.
"Do our URLs leave your platform?"
With the licensed database, lookups run inside your own systems and the answer is no. On the API path, send only the domain, never the full URL, path or query string.
"Is our data shared with other tenants?"
The shared cache holds only public domains and their categories. Tenant policies, exceptions and logs stay in each tenant's own space.
"Where is the data processed?"
Database files can be deployed in every region you operate in, including air-gapped ones, which simplifies data residency answers.
"How long do you keep lookups?"
Set a short retention for lookup logs and keep only aggregates for quality work. Document it in your data processing terms.
Common integration mistakes
These problems come up often in SaaS integrations. Each one is cheap to avoid at design time and expensive to fix after launch.
Looking up full URLs
Sending every unique URL multiplies calls and leaks paths that may carry personal data. Look up domains and hosts only.
No cache lifetime
A cache that never expires keeps old labels forever. A cache that expires in minutes burns quota. Pick days, and shorten it for safety paths.
Blocking on the first label only
Safety rules that check only the top-ranked category miss mixed sites. Check all returned labels for block rules.
Hiding the reason
A link that silently disappears creates support tickets. Show the category that triggered the rule and how to request an exception.
Treating shorteners as the destination
A short link's category says nothing about where it goes. Resolve the redirect, or apply a stricter rule to URL Shorteners.
One global policy
Customers differ. A rule that suits a school platform will frustrate a marketing agency. Keep policies per tenant from the first release.
Evaluation checklist before you commit
Run these checks on any URL category source before you build a feature on it. They take a few days and save months of rework.
| Check | How to run it | What good looks like |
|---|---|---|
| Coverage on your traffic | Export a week of domains from your product and look them up in bulk | Very few unknowns among domains your users touch often |
| Fit of the categories | Map your feature's rules to the category list | Every rule maps to one or more categories without workarounds |
| Freshness | Look up domains registered in the last few days | New domains already carry labels |
| Speed | Time lookups on your interactive path | Within your latency budget with the cache warm |
| Licence fit | Describe exactly what customers will see | The licence covers that use in writing |
Licensing for SaaS products
Showing categories or decisions to your own customers is an embedded use of the data. That calls for an OEM licence, explained on the OEM licensing page.
OEM / embedded
The data powers features in your SaaS product. Customers see categories for URLs in their own account and the decisions your product makes.
Most SaaS integrationsInternal use
For analytics or security work inside your own company, with nothing shown to customers.
Internal teamsNot allowed
Resale or publication of the raw data, such as a bulk export feature that hands customers the domain list.
Applies to every licenceURL category API questions from SaaS teams
Should a SaaS product use a URL category API or a licensed database?
Start with the API if volume is low or you are validating a feature. Move to the licensed database, with the API for misses, once lookups reach steady high volume, latency matters, or customers ask that their URLs never leave your platform.
Can we show categories to our own customers?
Yes, under an OEM licence. Your customers can see the category for URLs in their own account. What is not allowed is resale or publication of the raw data, such as an export of the full domain list.
How many categories does the API return?
Domains are classified into 59 web filtering categories, and a domain can carry several categories ranked by how much of the site they describe. The licensed file also includes the IAB Content Taxonomy.
Does the API support batch lookups?
Yes. There is a single lookup endpoint and a batch endpoint that accepts up to 1,000 domains per request. Details are in the API documentation.
How fresh is the data behind the API?
About 300,000 newly registered domains are categorized every day, rechecked after a week and after a month, and all other domains get regular monthly refreshes. Database licensees receive the same updates as daily delta files.
How do we keep API costs predictable?
Normalize URLs to domains, deduplicate, cache answers for a set period, and batch lookups. Most SaaS products see heavy repetition across tenants, so a shared cache removes most calls.
Does the API return a country for each domain?
Yes. Responses include a country code associated with the domain, which SaaS products use for regional reporting and geographic rules alongside the categories.
What should our product do with a domain that has no category yet?
Pick a default per feature. Safety features often warn or hold the link for review, while analytics features place it in an uncategorized group and retry later, since new domains are categorized daily.
Can we run the database in several cloud regions?
Yes, within the terms of your licence. The files run inside your own systems, so you can deploy copies in each region you operate in, including air-gapped environments.
Can we combine the categories with our own labels?
Yes. Many SaaS products add their own tags on top, such as customer-specific risk labels or internal partner lists. Keep them in a separate table keyed by domain, apply them after the category lookup, and let your own labels win where both exist.
Add URL categories to your product
Tell us which feature you are building and your expected lookup volume. We will suggest the API plan or database licence that fits.
Talk to us OEM licensing