Twelve concept pages, arranged as a reading path rather than an alphabetical list. Start at the definition, work through why domain-level controls break, then the risk taxonomy, then the governance frame. Each stands alone if you already know the ground.
An AI agent allowlist is a policy that decides, per URL and before the request, whether an autonomous agent may go there. It works at the page level rather than the domain level, because the same domain hosts both the page you want the agent reading and the page you never want it touching.
Enforcement stacks in four layers: a curated host list of roughly 60 high-value or dangerous hosts; the page-type database covering 28 types across 40M+ domains; roughly 40 egress rules that classify risky URL shapes on any domain at all; and default-deny for everything still unclassified. The pages below take those apart one at a time.
Definitions and the basic shape of the control. Read these two first if the category is new to you.
The core argument. If you only read one chapter, read this one: it is the reason page-type data exists as a product at all.
The risk side. What the failure modes are, how an incident unfolds step by step, and why telling a model not to do something is not a control.
Mapping the control onto security frameworks your organization already uses, and the process that keeps it current once it is live.
Several high-profile 2026 incidents involved AI agents escaping their task or being steered off it: the Hugging Face breach, the DseWiki wiki hijack, a covert channel through JFrog Artifactory, and third-party account takeovers. Our analysis shows the documented attack steps in almost all of them would have been denied at the request by the database and egress rules together. It is the clearest way to see the theory on this page do actual work.
The 2026 agent incidents, preventedTerms you do not recognize are defined in the glossary, and the FAQ answers the product and licensing questions these pages deliberately skip. To see the concepts as actual data, the free sample CSV has 100 real domains with every column, and the page-types database page explains the coverage. A related but separate problem — which AI tools your staff may use — is covered at aitoolsblocklist.com.
The concepts here are deliberately implementation-free. These three sections are where they turn into code, policy and a purchase decision.
Design the policy, wire the hook into your framework, then operate and audit it.
Compare — 8robots.txt, bot detection, proxy filtering, prompt guardrails, and building the data yourself.
Use cases — 15Fifteen agent jobs with the allow and deny sets each one actually needs.
