URL category data for link moderation in chats, communities and games
Every link a user posts is a decision: deliver it, warn about it, hold it for a moderator, or remove it. This guide is for trust and safety and product teams at chat apps, forums, game platforms and marketplaces who want to make that decision consistently, in milliseconds, from the category of the site behind the link.
Five actions a link policy can take
Good link moderation is not a yes or no. Most platforms use a ladder of actions and pick one per category and surface.
Deliver
The link goes through untouched.
Warn
The reader sees a notice before leaving the platform.
Hold
A moderator approves the message first.
Strip
The message posts without the link.
Block
The message is refused and the sender told why.
Categorize every posted link before the message is delivered
With our database loaded in your own systems, a link's category is a local lookup that takes a fraction of a millisecond. That is fast enough to check every message on every surface, with no call to an outside service and no message content leaving your platform.
- 58 categories to map onto your actions per surface.
- Several ranked categories per domain for nuanced rules.
- Daily new domains, so fresh scam sites do not stay unknown for long.
Policies by surface
The same link deserves different treatment in a children's game chat and a developer forum. A starting matrix, using our category names:
| Surface | Block or strip | Hold or warn | Deliver |
|---|---|---|---|
| Children's game or app chat | Adult, Dating, Gambling, Lingerie, VPN / Proxy, Social Networking, unknown domains | Everything not on an allow list | An allow list of game, school and help sites |
| Teen community | Adult, Dating, Gambling, Illegal Drugs, Weapons | URL Shorteners, Cryptocurrency, unknown domains | Most other categories |
| General adult community | Adult outside age-gated spaces | URL Shorteners, Parked / For-Sale, unknown domains | Most other categories |
| Marketplace messages | Links that move payment or chat off the platform, where your rules forbid it | Email and Messaging, Cloud Storage / File Hosting | Shopping and product pages |
| Professional or developer forum | Adult | URL Shorteners | Developer Tools, Computers & Technology, Education |
The link pipeline, step by step
Six steps between a message arriving and a decision. Each one is cheap; together they handle the tricky cases.
Extract
Find every link, including ones without "https://" and ones split across lines.
Normalise
Lower-case the host, decode look-alike characters, drop tracking parameters.
Expand
Follow shorteners and redirects server side, with a limit on hops.
Reduce
Trim the final host to its registrable domain with a public suffix list.
Look up
Read the domain's categories from the local database; cache the result.
Decide
Apply the surface's policy and log the action and the reason.
Keep the threat feed check beside step 5. Categories say what kind of site a link leads to; a threat feed says whether a specific address is known to be phishing or malware. The two answer different questions.
Six domains that show up in chats every day
Real rows from our data. Each category maps to an action in the matrix above, and none of them needs a moderator to look at the page.
| Domain | Categories in our data | Typical action |
|---|---|---|
| discord.gg | Email and Messaging | Deliver; hold on children's surfaces |
| t.me | Email and Messaging, Social Networking | Deliver for adults; strip in children's chat |
| linktr.ee | URL Shorteners | Expand and categorize the links behind it |
| tinyurl.com | CDN / Hosting / Infrastructure, URL Shorteners | Expand; warn if expansion fails |
| iplogger.org | URL Shorteners, Computers & Technology | Treat as a shortener: expand or strip |
| mega.nz | Cloud Storage / File Hosting | Warn; files can hold anything |
Edge cases that trip up link filters
Spammers and abusers know where simple filters fail. Six cases to design for from the start.
Shortener chains
One short link leading to another. Limit hops, categorize the final domain, and treat an endless chain as a warning sign.
Link-in-bio pages
A single page listing many links. Its own category says little; the links on it are what matter.
Look-alike characters
Hosts that use similar-looking letters from other alphabets. Normalise and compare before you look up.
Brand-new domains
Scam sites often post links within days of registration. Unknown domains need their own rule per surface.
Links in images
Text in screenshots and QR codes bypasses link parsing. Image text recognition feeds the same pipeline.
Edited messages
A harmless link edited into a harmful one after delivery. Re-run the check on every edit.
Using several categories per domain
Popular domains often carry more than one category, ranked by how much of the site each describes. Two rules make that useful rather than confusing.
Strict categories: any match counts
- Adult, Gambling, Dating, Illegal Drugs, Weapons on young surfaces
- VPN / Proxy where you prevent filter evasion
- A domain with Adult anywhere in its list is treated as adult
Broad categories: the main one decides
- Shopping, Business, Education and similar labels
- Read the first category to describe the link in a warning
- Avoid blocking a news site because one section sells things
More on how often domains carry several categories, with counts from our data, in multi-category websites.
How the database narrows the gap for new domains
About 300,000 newly registered domains receive a category every day, and each is checked again after a week and after a month as sites fill in. Daily delta files bring them into your copy, and the included lookup API answers for anything not yet loaded.
Online safety rules and link moderation
Laws such as the UK Online Safety Act and the EU Digital Services Act expect platforms in scope to assess risks and take proportionate measures, with extra care for children. Consistent link rules help in three ways.
Describable
"Adult and gambling links are removed from under-18 surfaces" is a measure a risk assessment can state plainly.
Consistent
The same link gets the same action every time, which is hard to achieve with moderators alone.
Measurable
Counts of actions by category and surface show the measure working over time.
Reviewable
Logged reasons let you answer user appeals and regulator questions with specifics.
This is general information, not legal advice. Which duties apply depends on your service, your users and where they are.
Measuring whether link moderation works
Four numbers to watch each month, by surface.
| Measure | What it tells you | Watch for |
|---|---|---|
| Actions by category | Which kinds of links your users post | Sudden rises in one category, often a spam wave |
| Share of unknown domains | How fresh the threats on your platform are | A growing share means new-domain campaigns |
| Appeals upheld | Where rules are too strict | A category with many upheld appeals needs a softer action |
| User reports on delivered links | Where rules are too loose | Reports clustering in a category you deliver |
What users should see
The wording of a warning decides whether users trust the rule or fight it. Short, specific messages work best, and the category gives you the specific part.
Warning before leaving
"This link goes to a file sharing site outside our app. Only open it if you know the sender."
Held for review
"Your message has a link to a gambling site, so a moderator will check it before it appears."
Link removed
"We removed a link to an adult site from your message. The rest was posted."
Unknown site
"We could not check this new site yet. The link will appear once it has been reviewed."
Blocked message
"Links to proxy and VPN services are not allowed in this space. You can post your message without it."
Appeal route
"Think we got this wrong? Tap here to ask a moderator to look again."
Rolling out link rules in four weeks
Start by measuring, then act on the clear cases, then tune. Users notice rules less when they arrive this way.
| Week | Do | Learn |
|---|---|---|
| 1 | Categorize every posted link, take no action, record results | Which categories your users actually post |
| 2 | Turn on blocks for the clear cases on young surfaces | How many messages are affected and how users react |
| 3 | Add warnings and holds for shorteners, file hosts and unknown domains | Moderator workload and appeal rates |
| 4 | Review appeals and reports, adjust actions per category | Where rules are too strict or too loose |
Moderating links by hand, or with licensed category data
The same stream of messages, two ways.
| Task | Moderators and keyword lists | Licensed category data |
|---|---|---|
| Deciding on a link | A person opens the page | A local lookup returns its categories |
| Speed | Minutes to hours in a queue | Before the message is delivered |
| Consistency across surfaces | Depends on who is on shift | One policy matrix, applied every time |
| New domains | Unknown until someone looks | About 300,000 categorized every day |
| Message privacy | People read user messages | Only domain names are looked up, in your systems |
Link moderation with URL categories: frequently asked questions
How can a chat or community app moderate links?
Extract each link, reduce it to its registrable domain, look the domain up in a category database, and apply a policy per surface: allow, warn, hold for review, remove the link, or block the message.
Is a domain category enough to stop phishing links?
No. Categories tell you what kind of site a link leads to, such as adult, gambling or a URL shortener. Known phishing and malware addresses change by the hour and need a threat feed alongside the category data.
How should URL shorteners be handled?
Expand them server side and categorize the final destination. If expansion fails or the chain is long, hold the message or show a warning instead of passing the link through.
How fast is a category lookup?
With the database loaded in your own systems, a lookup is a local key search and takes well under a millisecond, so it can run on every message before delivery.
What should happen to links whose domain is unknown?
Treat them by surface. On children's surfaces hold or strip them; on adult communities allow them with a light warning, and recheck later when the domain has been categorized.
Can category data support online safety obligations?
It can be one of the measures. Platforms with duties under laws such as the UK Online Safety Act or the EU Digital Services Act document their risk measures; consistent link rules by category are easy to describe and evidence.
Database licences start at $999 for the 1 million most popular domains, and OEM licences for platforms are quoted per deployment. Tell us your message volume and surfaces, and we will scope the files and the licence.
Put a category behind every link your users post
Tell us about your platform, its surfaces and its users' ages. We will suggest the categories, the delivery and the licence for your link checks.
Domain names in the tables identify services discussed for illustration only. Web Filtering Database is not affiliated with, sponsored or endorsed by their owners.