A content-category taxonomy answers "what kind of site is this" — software, finance, news, retail. A page-type map answers a completely different question: "where on this specific site is the login, the checkout, the pricing page." Agent policy needs both answers, at the same time, on the same request, and treating either one as sufficient on its own leaves a real gap in coverage.
Category alone would not have caught the DseWiki hijack. A wiki domain classified under a benign "reference" or "community" category still exposes a wiki_edit endpoint that page-type policy denies specifically. Several 2026 incidents crossed exactly this gap — our database and egress rules would have prevented almost all of them.
Web categorization is a well-established discipline: classify a domain into a content taxonomy such as IAB's 700+ categories across four tiers, or a coarser web-filtering scheme, so that policy can be written in terms of "software," "finance," "adult content," or "gambling" rather than in terms of individual URLs. It answers a question about the site as a whole, and it answers it well — a single category label, assigned once per domain (with periodic refresh), tells a filtering or research system what kind of content or business the domain represents.
That label says nothing about which specific page on the domain an agent is about to request. A domain categorized as "Software" carries that label whether the URL in question is its public blog post, its documentation, its pricing page, or its login form. Category answers "what is this site," uniformly, across every page the domain serves. It was never built to answer "is this particular URL a place my agent should be submitting credentials," because that question operates at a level of granularity category was not designed to reach.
Page-type classification answers exactly that narrower question, and only that question. It says nothing about whether the domain overall is a bank, a blog, or a SaaS vendor; it says, for one verified URL on that domain, which of 28 functional types the page serves — login, checkout, pricing, documentation, and so on. A page-type classifier has no opinion on whether the site as a whole belongs in a "Business" or "Technology" category, because that is not the question it was built to answer either.
Put the two side by side and neither one is wrong or redundant; they simply answer different halves of the same policy question. A team that has only a category feed can write a rule like "allow research agents onto Software-category domains," but that rule has no way to keep the same agent off that domain's login page. A team that has only a page-type map can deny logins and checkouts everywhere, but has no way to additionally say "and be more cautious across Finance-category domains specifically." Combined, the two produce a policy neither one could express alone.
The mismatch is easiest to see in scale terms. A domain-level category assignment is, by construction, one label (or one small set of labels across tiers) per domain, which means a site with ten thousand pages gets the same category value on every one of them. Page-type coverage runs in the opposite direction: it is deliberately sparse and specific, naming only the handful of pages per domain — typically well under two dozen — that correspond to one of the 28 defined functional types, and saying nothing about the other several thousand pages that do not fit any of those types. Neither schema was built to do the other's job at the other's resolution, and no amount of refining one closes the gap the other is meant to cover.
One label (or set of labels across IAB v2/v3 Tier 1–4, plus a 59-category web-filtering scheme) assigned per domain, describing the nature of its content or business.
28 verified page types per domain, each a specific URL that answers a specific functional question about that one page, independent of what the domain as a whole is about.
| Dimension | Category taxonomy (IAB / web-filtering) | Page-type schema (28 types) |
|---|---|---|
| Question answered | What kind of site is this, overall | What function does this specific URL serve |
| Unit of classification | The domain (sometimes subdomain) | The individual page |
| Depth | Up to 4 tiers (IAB), or one of 59 filtering categories | 28 named types across identity, commerce, content-write, and research/read groups |
| Distinguishes login from blog on the same domain? | No — both carry the same domain-level category | Yes — each has its own verified URL and type |
| Distinguishes a bank's site from a blog? | Yes — that is exactly its job | No — a login page is a login page regardless of the domain's business |
| Typical policy use | "Restrict research agents to Business/Software domains" | "Deny checkout, cart, and identity pages everywhere, allow pricing and docs" |
| Refresh cadence | Periodic, per-domain, as content and business focus shift | Verified per URL, refreshed alongside the domain database |
| What each one misses alone | Any distinction between pages on the same domain | Any signal about the domain's overall subject matter or industry |
The two schemas are not competing standards for the same job. They are complementary axes on the same domain record: every one of the 40 million+ domains in the database carries both an IAB category, a web-filtering category, and a set of verified page types, so a policy can combine "what kind of site" with "what kind of page" in a single rule.
Consider a compliance-monitoring agent tasked with tracking terms-of-service and privacy-policy changes across a company's vendor list, plus a wider set of Financial Services-category domains for regulatory-change awareness. A category-only policy can scope the second half of that task — "watch Financial Services domains" — but has no way to say "and only touch their legal and press pages, never their login or account pages," because category does not resolve to individual page types at all. A page-type-only policy can enforce the safety half — "allow legal and press, deny login and account" — on any domain, but has no way to scope the monitoring specifically to Financial Services as a vertical, because page type carries no industry signal.
| Rule expressed | Category alone | Page type alone | Combined schema |
|---|---|---|---|
| Scope monitoring to Financial Services domains | possible | not possible | possible |
| Deny login/account pages on those domains | not possible | possible | possible |
| Allow legal/press pages on those domains only | partial | partial | possible |
| Apply the same safety rule everywhere, regardless of industry | not possible | possible | possible |
Only the combined schema can express all four rules in the same policy document. This is not a marginal improvement; it is the difference between a policy that can be scoped by industry and a policy that cannot, and between a policy that is safe by page type and one that only sounds safe because it never had to name a specific dangerous page.
The gap shows up most clearly during an audit or compliance review, when someone asks a two-part question: "which domains was the agent authorized to visit, and what could it do once it got there." A category-only record answers the first half well and cannot answer the second at all — it has no page-level field to point to. A page-type-only record answers the second half precisely and has nothing to say about whether the domain list itself was appropriately scoped to the task's industry or subject matter. Only a record carrying both fields lets a reviewer trace a single navigation decision back to both the domain-level reason it was in scope and the page-level reason it was allowed or refused, which is the level of detail most governance reviews are actually asking for once they get past the first question.
A market-research agent is scoped to survey News and Business-category domains for coverage of a product category, pulling article and press content while staying off anything transactional or credentialed. Here is how category and page type resolve together on four representative requests.
| Request | Category | Page type | Result |
|---|---|---|---|
| /2026/09/market-report on Domain A | News | blog | allow — in scope, safe type |
| /press/product-launch on Domain B | Business > Software | press | allow — in scope, safe type |
| /subscribe on Domain A (paywall prompt) | News | subscribe | flag — in scope, action type |
| /account/login on Domain C | Retail — out of task scope entirely | login | deny — wrong category AND unsafe type |
The fourth row shows both schemas agreeing for different reasons: Domain C is out of scope on category grounds (it is not News or Business), and its login page would have been denied by page-type policy regardless of category. The third row shows why the two schemas cannot be collapsed into one: subscribe is in-scope by category (a News domain, exactly the kind of site the task cares about) but is still an action page type, so it is flagged for human sign-off rather than silently allowed or silently denied. Category told the agent this domain matters; page type told it exactly how carefully to proceed once there.
A policy that allows an agent onto any domain in a "trusted" category, with no further check, treats the entire domain as equally safe — its blog post and its account-settings page get the same verdict, because category cannot distinguish between them. The agent proceeds past a login or checkout with no additional resistance, since nothing in the category schema was ever built to flag that specific page.
Category answers "should this domain be in the agent's task at all," and page type answers "is this specific page one the agent may act on." A domain passing the first test still has its login, checkout, and upload pages denied by the second, independent of how trustworthy the domain's overall category appears.
The free sample CSV shows this directly: every one of its 100 rows carries a domain alongside its page types, its IAB v2 and v3 categories across all four tiers, its web-filtering category, personas, OpenPageRank, country, and popularity rank groups — one record, two schemas, no need to join separate datasets.
See the full column reference, including exact IAB tier structure and the web-filtering category list, on the page-types database page, or pull all twelve columns directly from the free 100-domain sample. Teams building their own policy engine on top of this record typically load it once into whatever table or index already backs their existing category-based rules, then add a second lookup keyed on the exact URL rather than the domain, so the two checks can run in the same request path without a separate service call for each schema.
Deny login, signup, checkout, cart, upload, and the rest of the identity and commerce groups by default, on any domain, before category enters the decision.
Add "and only proceed on Financial Services / Software / News category domains" as a narrowing filter for the specific task, never as a replacement for the page-type baseline.
A "trusted" category is not a reason to allow a login or checkout page. The safety rule wins regardless of what industry the domain belongs to.
Record the category and the page type on every navigation decision, so a later review can see both why a domain was in scope and why a specific page was allowed or denied.
A single check against the database or API returns the page type for enforcement and the domain's category fields for scoping, in one response:
Full-field responses, including IAB and filtering categories per domain, are part of the licensed database rather than the lightweight per-URL check endpoint; see the API docs for request shapes and the pricing page for lookup and license tiers.
For the same category taxonomy applied to blocking human access to risky AI tools rather than agent navigation, see Web Filtering Database, the sibling product covering 100M+ domains by filtering category.
28 page types, 700+ IAB categories, 59 filtering categories, 40M+ domains. Start with the free sample.
