AI Agent Allowlist
Home Page-Types Database Agent Guardrails 2026 Incidents API Docs Pricing
Resources
Use Cases Industries & Buyers Learn: Core Concepts Implementation Guides Comparisons Schema & Data Reference FAQ Glossary
Why It Matters
2026 Agent Incidents Category Targeting Database Refreshes Contact Customer Login
Download Free Sample
the full taxonomy, exhaustively

All 28 Website Page Types, Explained With Examples

Every meaningful action an agent (or a human) can take on a website happens on a specific kind of page, and those kinds fall into a manageable, finite taxonomy: 28 page types across four groups. This page walks through every one of them individually — what it is, a concrete real-world example, and how a typical agent policy treats it — in more depth than the summary table on our page-types database overview.

28Page types, in four groups
4Groups: identity, commerce, content-write, research
8Action types added in the current reclassification cycle
40M+Domains classified against this taxonomy
The organizing idea

Grouped by what the page lets someone do

The 28 types are not an arbitrary list — they group into four categories based on the kind of action a page enables, which is also the single most useful signal for deciding whether an unattended agent should be allowed there. Twenty of the twenty-eight are the original, broadly available set; the remaining eight are "action types" — pages where an agent does something rather than reads — shipping with the current reclassification cycle, so not every domain has them populated yet.

Read the groups below in the order they are presented, because the order itself is meaningful: it runs from the pages an unattended agent should almost never reach (identity) through the pages it should reach only with explicit authorization (commerce, content-write) to the pages that make up the bulk of what a well-scoped research task actually needs (research/read). If you only remember one thing from this page, remember that ordering — it is the same ordering a sound default policy follows, group by group, before any domain-specific exception is layered on top.

It is also worth being explicit about what this taxonomy is not trying to do. It does not classify a site's subject matter or industry — that is the separate job of the 700+ IAB content categories and the 59-category web-filtering taxonomy carried in the same database record. Page type answers "what can happen on this specific page"; content category answers "what is this site generally about." A mature policy checks both, because a page can be low-risk by content category and still be a page type worth denying, or vice versa.

01

Identity — 3 types

Pages that create or recover a specific person's access to an account
Deny by default

login

The authentication form for an existing account. Often lives on a separate identity subdomain (accounts.example.com) or third-party identity provider rather than the main site.

Example: a SaaS product's customer sign-in page, separate from its marketing site
Deny by default (action type)

signup

The account-registration form. An unattended agent creating a new identity on a third-party service is very rarely the intended outcome of a research task.

Example: a "start your free trial" registration form
Deny by default (action type)

password_reset

The credential-recovery flow. A legitimate use for a human account owner; an agent reaching it unattended is a strong incident signal, not a normal research step.

Example: a "forgot your password" email-reset form
02

Commerce — 5 types

Pages that price, add to, or complete a purchase
Typically allowed

pricing

The page agents look for most in research tasks. A verified URL turns a multi-step crawl-and-guess into a single lookup — zero browsing, minimal token cost.

Example: a "/pricing" or "/plans" page listing tiers and costs
Typically allowed

product

An individual product or listing page describing a specific item, plan, or SKU — useful for competitive and catalog research without touching any transaction flow.

Example: a single product detail page on a retailer's site
Deny unless authorized (action type)

cart

Where selected items accumulate before checkout. Reaching it is usually a precursor to a transaction an unattended agent should not be initiating.

Example: an e-commerce shopping-cart summary page
Deny unless authorized (action type)

checkout

The purchase-completion flow — the clearest line between research and an unintended order. Denied unless an agent is explicitly authorized to transact on that specific domain.

Example: a payment and shipping-details checkout form
Deny unless authorized (action type)

subscribe

A recurring-payment or newsletter sign-up flow. Financial subscriptions carry the same transaction risk as checkout; even a free newsletter signup creates an unintended account an agent should not open unattended.

Example: a "subscribe for $9/month" recurring billing form
03

Content-write — 4 types

Pages where content is published or a persistent record is created
Deny by default (action type)

post_create

The interface for publishing new content — a blog post, a forum thread, a listing. An agent with write access here can create public content in the organization's name without a human review step.

Example: a CMS "new post" editor or a forum's "start a thread" form
Deny by default (action type)

comment

A reply or comment submission field on an existing piece of content. Smaller in scope than post_create, but still a public write action taken under the organization's identity.

Example: a comment box under a blog article or support ticket
Deny by default (action type)

upload

Any file or dataset upload interface. This is the exact page type our incident analysis shows was the entry vector for a 2026 breach involving malicious dataset uploads.

Example: a dataset or file upload form on a hosting or collaboration platform
Deny by default

community

A community or discussion hub with its own posting and membership mechanics — broader than a single comment field, and often carrying its own write endpoints worth mapping separately.

Example: a product's community forum landing page
04

Research / read — 16 types

The safe reading surface most agent tasks actually need
Typically allowed

documentation

Technical or product documentation — the highest-value target for developer-facing and integration-research agents.

Example: a "/docs" site for an API or developer platform
Typically allowed

help_center

End-user support articles and how-to content, distinct from developer documentation.

Example: a "Help Center" or knowledge-base search page
Typically allowed

integrations

A directory of a product's supported integrations or partner connectors — useful for compatibility research.

Example: a SaaS product's "Integrations" marketplace page
Typically allowed

status

A live service-status or uptime page, typically on a dedicated subdomain that path-guessing (status.example.com) often gets wrong without verification.

Example: a "status.company.com" incident and uptime dashboard
Typically allowed

security

Security disclosures, vulnerability-reporting policy, or a trust-center summary — relevant to vendor due-diligence and compliance-monitoring tasks.

Example: a "/security" or ".well-known/security.txt" disclosure page
Typically allowed

contact

General contact information or a contact form — low-risk to read, though an agent submitting the form itself is a separate policy question from simply reading the page.

Example: a company's "/contact" page with an address and form
Typically allowed

about

Company background — who operates the domain, and since when — central to entity resolution and vendor research.

Example: an "/about" or "/company" overview page
Typically allowed

leadership

Executive or leadership team pages, useful for sales-intelligence and org-mapping research.

Example: an "/about/leadership" or "/team" page with executive bios
Typically allowed

careers

Open roles and hiring information — a common target for recruiting-sourcing and growth-signal research.

Example: a "/careers" job listings page
Typically allowed

blog

The company's editorial or blog content — a core target for content-research and market-research agents.

Example: a "/blog" article index
Typically allowed

press

Press releases and media coverage — useful for tracking company news and announcements at scale.

Example: a "/press" or "/newsroom" release archive
Typically allowed

events

Webinars, conferences, and other scheduled events a company is hosting or attending.

Example: an "/events" calendar page
Typically allowed

partners

Partner and channel directories, useful for ecosystem and channel-research mapping.

Example: a "/partners" directory of resellers or technology partners
Typically allowed

case_studies

Customer proof-point pages — structured evidence of who uses a product and how, valuable for competitive and sales research.

Example: a "/customers" or "/case-studies" success-story index
Typically allowed, read-only

legal

Terms of service, privacy policy, and related legal documents — a primary target for compliance-monitoring agents tracking policy changes over time.

Example: a "/terms" or "/privacy" legal-document page
Typically allowed

sitemap

The site's own navigation or XML sitemap — a structural reference rather than content, sometimes useful for an agent orienting itself on an unfamiliar domain.

Example: a "/sitemap.xml" or human-readable site-map page

This group is deliberately the largest, because it is the safe reading surface most agent tasks actually need. A well-designed policy allows almost all of it by default and reserves default-deny for the identity, commerce, and content-write groups above — which is exactly the four-group split this taxonomy is built around.

A handful of these sixteen still deserve individual attention rather than a blanket allow. The legal type, for instance, is read-only but sometimes carries a jurisdiction-selection or cookie-consent interaction that borders on a form submission; a strict policy may allow reading the page while still denying any interaction beyond a plain fetch. The contact type raises the same question in reverse — reading the page is harmless, but an agent that goes on to submit the contact form itself has crossed into a write action this taxonomy tracks separately once a domain's form-submission endpoints are mapped. Treat the sixteen as a strong default, not a blanket exemption from thinking about the specific task at hand.

A distinction worth understanding

Absent vs. guessed: why some domains show fewer than 28

Not every domain has all 28 page types, and that is expected rather than a data gap. A personal blog has no checkout because it has no commerce; a documentation-only open-source project may have no careers page because the maintaining organization does not list jobs there. When a page type genuinely does not exist on a domain, the database records it as explicitly absent — not as a guessed or assumed URL. This distinction matters for policy: an absent page type is a fact about the site, while a missing record from an incomplete crawl would be a data gap, and the two should never be treated the same way by a policy engine deciding whether to allow or deny.

This is also why the eight action types are described as "shipping with the current reclassification cycle" rather than universally present: as domains are re-verified against the newer action-type definitions, more records gain populated signup, checkout, upload and similar values. A record without one of the eight action types populated yet is a coverage-in-progress state, not a claim that the page type is absent from that domain — worth checking against the database overview if your policy depends on the distinction.

Applying the taxonomy

The same 28 types, two very different agent roles

A taxonomy only earns its keep once it is used to make different decisions for different situations. Here is how two common agent roles end up with very different allow lists, built from the exact same 28 page types.

A vendor-research agent tasked with evaluating potential software purchases needs almost the entire research/read group: pricing and product to understand offerings, documentation and integrations to assess technical fit, security and legal to support a procurement review, case_studies and leadership for vendor credibility checks. It has no legitimate reason to reach cart, checkout, or subscribe — its job is to inform a human's purchase decision, not to make one — and no reason to reach login, signup, or password_reset, since it is not meant to hold or create any account on the vendor's behalf. A policy for this role can allow essentially all of group four, deny all of groups one, two, and three, and be done.

An internal knowledge-base maintenance agent, by contrast, has a role built around exactly the page types the vendor-research agent should never touch: it may legitimately need post_create and comment to update internal documentation, and possibly upload to attach supporting files, because publishing internal content is its actual job. But it still has no legitimate reason to reach an external checkout page, and no reason to authenticate to third-party services outside a narrow, explicitly approved set of internal tools. The same content-write group that is denied by default for a research agent becomes exactly what a maintenance agent's policy needs to allow — scoped tightly to internal domains, not opened globally.

Neither policy required inventing new categories or writing custom logic per domain. Both were built by taking the same 28-type taxonomy and deciding, group by group and sometimes type by type, what this specific agent's job actually requires — which is the practical payoff of having a taxonomy exhaustive enough to describe both roles precisely, rather than a coarser scheme that would have forced one role's policy to be looser than it should be.

The upload page type, specifically, was a real 2026 entry vector

Our analysis of the 2026 Hugging Face breach shows the delivery mechanism was ordinary dataset uploads carrying a malicious payload — the upload page type from the content-write group above, plus the repo_write and object_storage_write egress rules, both deny by default. Every high-profile 2026 agent escape maps to a specific page type or rule in this taxonomy.

Every 2026 agent escape, mapped to the rule that stops it Read the Hugging Face breach case

The honest fine print — the same two assumptions we publish, plus two operational ones

  1. The policy engine must see every request — an agent with raw socket access or a second network path bypasses everything; enforcement belongs at the egress proxy/network layer, not only in an SDK hook.
  2. Default-deny must be on. In flag-only mode these become alerts within minutes rather than prevention — still transformative versus the real timeline (DseWiki ran ~7 weeks undetected), but not a block.
  3. For full URL+method matching on HTTPS you need to be the proxy or in-process hook — SNI alone shows only the host, which still catches the entire host-list layer.
  4. Policy can’t read intent inside a legitimately allowed action: an agent whose job is publishing packages keeps registry access. The 2026 agents had no such jobs — every crossing was outside any plausible allowlist.
Continue learning

Related pages

FAQ

Page-type taxonomy questions, answered

Why 28 types instead of a simpler scheme, like "sensitive" vs. "safe"?
A binary scheme cannot express the different policies organizations actually want — allow pricing but deny checkout, allow documentation but deny login, allow reading legal pages but deny submitting a comment. Twenty-eight specific types let a policy be as precise as the task requires, while still being a small enough, well-defined set to reason about.
Are the four groups (identity, commerce, content-write, research) a formal part of the schema?
They are a useful organizing frame for understanding the 28 types by what action each one enables, which is how this page and the database overview both present them. The database itself stores each page type as an individual field; grouping them is a policy-design convenience, not a technical restriction.
Do the "typically allowed" verdicts on this page apply to every organization?
No — they describe a common starting policy, not a fixed rule. A bank's compliance team may deny even blog and press pages for certain agents; a market-research firm may allow broader access than the defaults shown here. The database gives you the classification; your policy engine decides the verdict.
What happens if a URL doesn't match any of the 28 types?
It is recorded as unclassified for that domain, which falls to the egress rules and then default-deny under the standard four-layer model described in our allowlist definition — not silently allowed by default.
Are the eight action types available for every domain today?
Not yet universally. They are shipping with the current reclassification cycle, and records gain them as domains are re-verified. The original twenty types are present throughout the database already.
Can I see all 28 fields on real domains before committing to anything?
Yes — the free sample CSV covers 100 real domains with the page_types field and every metadata column the full database carries. Beyond the sample, the lookup API starts at $99/month.

See the full taxonomy applied to real domains

Download the sample CSV to see which of the 28 page types populate for 100 real, well-known sites.

Download the Sample