AI Agent Allowlist
Home Page-Types Database Agent Guardrails 2026 Incidents API Docs Pricing
Resources
Use Cases Industries & Buyers Learn: Core Concepts Implementation Guides Comparisons Schema & Data Reference FAQ Glossary
Why It Matters
2026 Agent Incidents Category Targeting Database Refreshes Contact Customer Login
Download Free Sample
how to evaluate before you buy

How to Evaluate Page-Type Data Quality Before You Sign Anything

Every page-type or classification vendor will tell you their data is accurate. The only way to know whether that's true for your use case is to test it yourself, on the free 100-domain sample, before a contract is on the table. This guide is that test plan: what "verified" actually means operationally, which metrics matter, which don't, and exactly how to spot-check the sample against reality in under an hour.

100Real domains in the free sample CSV, no signup required
28Page types you can spot-check per domain
<1hTime to run the manual spot-check procedure below
0Claims in this guide you should take on faith instead of checking
Start here

"Verified" is an operational claim. Make it prove itself.

Any classification vendor can say a URL is "verified." What that word means in practice varies enormously — from "a human clicked it once" to "an automated pipeline traced the site's own link structure and confirmed the destination" — and the difference matters enormously for a security control that's about to decide what an autonomous agent can and can't reach.

This distinction is easy to skip past in a sales conversation, because "verified" sounds precise regardless of what's actually behind it. A vendor whose "verification" is a human spot-checking a handful of top-1000 domains once, months ago, and a vendor whose classification pipeline traces the live link structure of every domain and re-derives the page-type map from what it actually finds are making very different claims, even though both might use the identical word on a landing page. You cannot tell these apart by reading marketing copy. You can tell them apart in about an hour with a free sample and a browser, which is the entire point of this guide.

What "verified" means for this database, specifically

A verified URL is a link discovered by tracing a domain's actual, live link structure — not a guessed or templated path — that was individually fetched and classified by a multi-stage pipeline into one of the 28 page types. An absent page type on a domain marked found: true is a verified negative: the pipeline checked and found no such page, not a gap the pipeline never got to. Both claims are testable against the sample, and the procedure below tests both.

What actually matters

Three questions worth answering, and one that isn't

Does the URL actually resolve to that page type?

Open a sample of the login, checkout, or pricing URLs listed for domains you recognize. Do they land on the page the row claims? This is the single most important check and the easiest to run yourself.

Is an absence actually an absence?

Pick a domain where a page type you'd expect (say, pricing on a SaaS product) is missing from the row. Check the live site: does it genuinely lack that page, or does it have one the classification missed?

How stale can a verified URL get?

Sites restructure. Ask what refresh cadence applies to your licensed tier or API plan, and whether a re-verification cycle is already in progress for newer page types — see the honest coverage note on action types below.

The question that isn't worth spending your evaluation time on: an aggregate "accuracy percentage" claimed by any vendor, including this one, without a methodology you can independently reproduce. A single top-line number invites exactly the kind of cherry-picking that makes it meaningless — test the specific domains and page types your own agents will actually touch, and treat aggregate accuracy claims from any classification vendor as a marketing number until you've verified a sample yourself.

Common evaluation mistakes

Where a quick evaluation goes wrong

Most bad buying decisions on classification data don't come from skipping evaluation entirely — they come from an evaluation that looks thorough but tests the wrong thing.

Only checking domains you already trust. Verifying the login URL for a handful of top-100 global brands tells you almost nothing about the long tail, which is exactly where a policy guessing paths instead of using verified URLs fails most often. Deliberately include a few domains you don't recognize in your spot-check, precisely because those are the cases a lower-quality classification pipeline is most likely to get wrong.

Testing only the page types you care about most. If your policy leans heavily on login and checkout denial, it's tempting to only verify those two page types and skip the rest. But if your policy also uses IAB or filtering categories as a condition — as the annotated schema in the policy file guide does — a category error can silently break a rule you never thought to test, in either direction: an over-broad allow or an over-broad deny.

Evaluating once and never again. A page-type database is not a static artifact; it is refreshed on a cycle, and coverage of newer page types like the eight action types genuinely changes over time as records are re-verified. An evaluation from months ago tells you what quality looked like months ago. Re-run at least the fast spot-check whenever a decision is actually riding on current coverage.

Confusing "the vendor's own dashboard says X% coverage" with an independent check. A coverage statistic reported by the vendor's own tooling, however accurate, is not the same evidence as opening the URLs yourself. It may well be correct — but the entire reason to run a five-minute manual check against a handful of domains you know is that it doesn't depend on trusting the reporting layer at all.

The spot-check procedure

Six steps, using only the free sample and a browser

01

Download the sample and pick 15 to 20 domains

Mix well-known domains you can verify from memory with a few you don't recognize, so you're not only testing the cases the vendor had the most incentive to get right.

02

For each domain, open every listed page-type URL

Confirm the URL loads and actually is the claimed page type — a login form for login, an actual pricing table for pricing, and so on. Note any mismatch.

03

For each domain, check one or two page types that are absent from the row

Visit the live site and look for that page type yourself. If the site clearly has one and the row doesn't list it, that's a coverage gap worth asking the vendor about directly.

04

Check the secondary fields, not just page_types

Do the IAB category, web-filtering category, language, and country fields match what you'd expect for each domain? These matter as much as page types if your policy conditions combine them, as in the policy schema guide.

05

Count how many of the sample's domains carry action-type page keys

Signup, password_reset, cart, checkout, upload, post_create, comment, and subscribe ship with the current reclassification cycle and aren't yet on every record. Use the sample to see the real, current proportion rather than assuming full coverage.

06

Run your actual policy logic against the sample

Load the CSV into the check function from the implementation guide and confirm your allow/deny/flag rules produce the split you expect on real rows before testing against a paid plan or licensed database at scale.

Illustrative code

A quick coverage script, sketched in Python

This sketch loads the sample CSV and reports basic coverage statistics — how many domains carry each page type, and what share of rows include at least one action-type key. It's illustrative only, meant to run in a few seconds against the 100-row sample as a starting point for your own evaluation script.

coverage_check.py — illustrative sketchPython 3
# Illustrative sketch — run against agent_allowlist_sample.csv
import csv
from collections import Counter

ACTION_TYPES = {"signup", "password_reset", "cart", "checkout",
            "upload", "post_create", "comment", "subscribe"}

def coverage_report(path: str) -> None:
  type_counts = Counter()
  rows_with_action_type = 0
  total = 0
  with open(path, newline="") as f:
    for row in csv.DictReader(f):
      total += 1
      present = [t for t in row["page_types"].split("|") if t]
      type_counts.update(present)
      if any(t in ACTION_TYPES for t in present):
        rows_with_action_type += 1
  print(f"Domains sampled: {total}")
  print(f"With at least one action-type key: {rows_with_action_type} ({rows_with_action_type/total:.0%})")
  for page_type, n in type_counts.most_common():
    print(f" {page_type}: {n}/{total}")

Run this before and after any refresh cycle you're evaluating, and compare the action-type percentage over time if you're deciding whether to wait for fuller coverage of the eight newer page types or start integrating now against the twenty that have been stable throughout.

Reading the sample columns

What each field is actually claiming

ColumnWhat to check it against
page_typesOpen each listed URL directly. It should load and visibly be the claimed page type, not a redirect to an unrelated page or a 404.
num_distinct_page_typesShould equal the count of types actually listed in page_types for that row — a basic internal-consistency check worth automating first.
languageCompare against the visible language of the site's main pages, especially for domains serving multiple locales.
IAB v2 / v3 Tier 1–4Does the assigned category match what you'd call the site if you were categorizing it yourself? Spot-check a few domains outside the obvious categories (news, e-commerce) where classification is harder.
Web Filtering CategoryCompare against the IAB category for the same row — they should tell a consistent story about what kind of site this is, even though they're two different taxonomies.
OpenPageRank, popularity rank groupsThese should roughly track your own intuition about how well-known each domain is; wildly inverted rankings for domains you know well are worth flagging.

None of these checks require special tooling — a spreadsheet, a browser, and the willingness to actually click through rows rather than skim them is the entire evaluation stack. The point isn't to build an elaborate audit; it's to replace "the vendor says it's accurate" with "I opened twenty rows and it held up," which is a categorically stronger basis for a purchasing decision.

From sample to purchase decision

What the evaluation should actually tell you to buy

The point of testing the sample isn't just confidence — it should change what you buy, not merely whether you buy.

If your spot-check holds up cleanly and your integration is a prototype or a workload where an external network call per lookup is acceptable, the self-serve lookup API is the fastest path from here, starting at $99/month for 90,000 lookups. If your evaluation surfaces a latency or data-residency requirement — an agent gateway that can't tolerate a network round-trip per navigation decision, or a deployment that can't call out to a third-party API at all — the same verified data is available as an on-premise database license, starting at $14,999 one-time for the 10M-domain tier, with an optional 30%-of-license-price annual refresh. Both delivery modes return the identical schema you just tested, so the evaluation you ran against the free sample carries over directly regardless of which one you choose. Full tier and volume details are on the pricing page.

One evaluation-driven decision worth making explicitly before you buy: if your policy depends heavily on the eight newer action-type page types, check the sample's current coverage of them (step 5 in the procedure above) and decide whether to integrate now against the stable original twenty and layer in action types as re-verification proceeds, or wait for broader action-type coverage first. Either is a reasonable choice — the point is making it based on what you actually measured, not on an assumption either way.

Before you sign anything

Evaluation checklist

Untested coverage is exactly what a hijacked agent exploits

The DseWiki hijack ran for roughly seven weeks on a wiki with legacy HTTP-GET write endpoints — a coverage gap in whatever traffic monitoring existed at the time went unnoticed for that entire window. Our analysis shows the same pattern applies to policy data quality: a page-type database nobody spot-checked against real domains is a database whose gaps you discover the same way, after the fact. Verifying coverage yourself, on the free sample, before it's load-bearing is the cheap version of that lesson.

Would your agents have been stopped? Check the incident analysis Read the DseWiki hijack analysis

The honest fine print — the same two assumptions we publish, plus two operational ones

  1. The policy engine must see every request — an agent with raw socket access or a second network path bypasses everything; enforcement belongs at the egress proxy/network layer, not only in an SDK hook.
  2. Default-deny must be on. In flag-only mode these become alerts within minutes rather than prevention — still transformative versus the real timeline (DseWiki ran ~7 weeks undetected), but not a block.
  3. For full URL+method matching on HTTPS you need to be the proxy or in-process hook — SNI alone shows only the host, which still catches the entire host-list layer.
  4. Policy can’t read intent inside a legitimately allowed action: an agent whose job is publishing packages keeps registry access. The 2026 agents had no such jobs — every crossing was outside any plausible allowlist.
Once you've evaluated it

Put verified data to work

If your evaluation also needs to cover domain-level risk rather than page-level classification — whether a whole site should be reachable at all, independent of which page type on it — the same spot-check discipline applies to aitoolsblocklist.com's domain categories, which answer a related but distinct question about the same URL.

FAQ

Evaluating page-type data, answered

What's the fastest way to sanity-check a page-type database?
Open 15 to 20 page-type URLs from the free sample CSV for domains you already know well, and confirm each one loads and is actually the claimed page type. This takes under an hour and catches the majority of real quality problems without needing any tooling.
How do I know if a missing page type is a real gap or a genuine absence?
Visit the live domain and look for that page type yourself. A found:true row with an absent page type should mean the classification pipeline checked and found nothing — if the live site clearly has one the row doesn't list, that's a coverage gap worth raising directly.
Should I expect every domain to have all 28 page types?
No. Availability depends on the site — not every domain runs all 28 page types in the first place, and the eight action types (signup, password_reset, cart, checkout, upload, post_create, comment, subscribe) are shipping with the current reclassification cycle rather than present on every existing record yet.
Is an aggregate accuracy percentage a meaningful thing to ask a vendor for?
Treat it skeptically without a reproducible methodology. A single top-line number can be constructed to look favorable regardless of real-world performance. Testing the specific domains and page types your own agents will actually touch tells you more than any aggregate claim, from any vendor.
Does the sample CSV reflect the same data quality as the paid API or licensed database?
Yes — the sample is 100 real rows from the same underlying database and schema, not a separately curated demo set. What you verify in the sample is representative of what the same fields look like at full scale.
How often should I re-run this evaluation after purchasing?
Spot-check again after any refresh cycle relevant to your plan, and whenever your policy starts relying on a page type or category condition it didn't use before — a field you never checked closely is a field whose quality you're taking on faith.

Run the spot-check yourself, today

Download the free 100-domain sample and work through the six-step procedure above before you evaluate anything else.

Download the Sample