AI Agent Allowlist
Home Page-Types Database Agent Guardrails 2026 Incidents API Docs Pricing
Resources
Use Cases (15) Industries & Buyers (12) Learn: Core Concepts (12) Implementation Guides (15) Comparisons (8) Agent Security Guides (22) Market & Frameworks (9) Schema & Data Reference (6) FAQ Glossary
Why It Matters
2026 Agent Incidents Category Targeting Database Refreshes Contact Customer Login
Download Free Sample

AI Agent Page-Type Statistics: an open dataset

How common are login, pricing, documentation and careers pages across the web? Aggregate counts from a page-type database of 40,786,727 domains, the data behind page-level egress control for AI agents. Free under CC BY 4.0.

40.8Mdomains covered
16.4Mwith verified page types
20page types
3.26Mdomains with a login page

Share of domains with each page type

Share of the 16,448,576 domains with at least one verified page type. Login, the page type an agent should never submit to, is highlighted.

contact
70.4%
about
49.6%
legal
46.6%
product
39.4%
blog
30.7%
login
19.8%
help center
16.5%
pricing
13.8%
events
12.1%
careers
11.1%
case studies
10.0%
leadership
8.8%
sitemap
6.3%
partners
6.0%
press
5.6%
documentation
3.9%
security
3.2%
status
2.9%
community
2.8%
integrations
1.2%

What is in the release

The page-type database behind the AI agent allow list records, for each domain, the verified URLs of its login, signup, pricing, documentation and other pages. An agent policy can then allow reads and deny writes at page level instead of trusting or blocking a whole site. This release publishes the aggregate picture: how often each page type occurs, how many page types a domain typically has, how that varies across 40 site languages and 60 top-level domains, which page types appear together, where the pages live (subdomains, query strings, path depth), and the categories of the separate host list. It also runs the 40 URL rules of the allow list on every verified URL: a rule matches 0.21% of read pages such as documentation or pricing, most of them registration or login pages filed under a read type (about 0.05% are genuine false positives), and 46% of login pages. The database itself stays commercial; besides the aggregates, the release includes the 100-domain sample with verified URLs.

Use it to estimate how much of the web exposes login or upload pages to autonomous agents, to size page-level policies, or as a reference distribution for web-measurement research.

page-type-prevalence.csvDomains with a verified URL for each of the 20 page types, 1 KB
Download
page-types-per-domain.csvHow many page types each domain has, 1 KB
Download
languages.csvDomains per site language (40 languages), 1 KB
Download
language-page-types.csvPage-type prevalence for all 40 languages, 25 KB
Download
page-type-cooccurrence.csvHow often two page types occur on the same domain, with conditional shares and lift, 10 KB
Download
tlds.csvThe 60 largest top-level domains, with the share that has page types, 2 KB
Download
tld-page-types.csvPage-type prevalence per top-level domain, 38 KB
Download
url-structure-by-page-type.csvWhere each page type lives: HTTPS, subdomains, query strings, path depth, 2 KB
Download
rule-layer-by-page-type.csvThe 40 URL rules run on every verified URL: match rate per page type, 2 KB
Download
rule-layer-first-rule.csvWhich rule matched first, per page type, 12 KB
Download
sample-100-domains.csv100 domains with their verified URLs, categories and popularity group, 95 KB
Download
host-list-categories.csvHost-list categories with default action and number of hosts, 1 KB
Download
README.mdDataset description, 6 KB
Download
DATA_DICTIONARY.mdColumns explained, 4 KB
Download
LICENSE.txtCC BY 4.0 licence, 1 KB
Download
CITATION.txtHow to cite, 1 KB
Download
CHANGELOG.mdVersion history, 1 KB
Download
manifest.jsonFile sizes and checksums (JSON), 3 KB
Download
SHA256SUMSChecksums, 2 KB
Download

Cite this dataset

Version 1.1.0, database snapshot of September 2026. DOI 10.6084/m9.figshare.34028775.

Alpha Quantum (2026). AI Agent Page-Type Statistics (Version 1.1.0). AI Agent Allow List. https://doi.org/10.6084/m9.figshare.34028775

Limitations: counts describe the database at one point in time, not the live web; a missing page type means no verified URL was found; labels come from an automatic classifier. Related open data: the AI tools taxonomy dataset and the web filtering research dataset.