Every agent platform tells its customers the model won’t submit credentials, complete a checkout, or post publicly on their behalf. After 2026, buyers stopped accepting that as a sentence in a prompt. They want to know what happens when the model is wrong — and the answer that survives a security review is a deterministic, external policy check, not a system message. AI Agent Allowlist is the data layer platforms embed to make that check possible, on 40 million+ domains, without building the crawl themselves.
OEM licensing means the page-type map, the egress rules, and the host list ship inside your product under your own brand — your customers never know it came from us unless you tell them.
Your customers were exposed to this. The 2026 agent incidents — the Hugging Face breach, the DseWiki hijack, the JFrog Artifactory covert channel, third-party account takeovers — all ran through platforms whose agents had unrestricted web access. Our database and egress rules would have denied almost every step, before the request left.
Before 2026, “the agent is instructed not to” was an acceptable answer in a sales call. It no longer is. Enterprise security teams evaluating an agent platform now ask, specifically, what mechanism sits between the model and a login form, a checkout page, or a comment box — and whether that mechanism is inside the model’s own reasoning or outside it. Platforms that can only point to a system prompt lose the deal to one that can point to a policy engine.
This shift matters most for platforms selling into regulated or security-conscious buyers — financial services, healthcare, government, and any enterprise with a formal vendor-security review. Those buyers increasingly write “deterministic, model-independent navigation controls” into their procurement requirements directly, sometimes without knowing that phrase maps to a specific, licensable dataset. Platforms that already have an answer close those cycles faster; platforms without one spend the review explaining why a prompt should be trusted instead.
Reproducing this data means traversing the live link structure of tens of millions of domains, individually classifying the links you find, and re-verifying the result on a schedule — because the web keeps moving. Platform teams that have priced this out generally conclude it is not their core product.
A traversal covering 40M+ domains and the billions of links inside them is a standing crawl operation, not a script — ongoing compute, IP reputation management, and rate-limit courtesy at scale.
Telling a login page at /welcome or /portal apart from a marketing page takes a multi-stage model pipeline tuned specifically for page-type detection, not a general-purpose classifier bolted on.
Sites redesign, domains change hands, roughly 300,000 new domains appear per update cycle. A one-time internal crawl is out of date within a quarter; keeping it current is the recurring cost nobody budgets for up front.
Page type alone under-specifies policy. Pairing it with 700+ IAB categories and a 59-category filtering taxonomy per domain, consistently, is a second classification system stacked on the first.
The write-surface rules and curated dangerous-host list that catch the long tail — wiki edits, WebDAV, registry admin panels — are a separate research effort most platform teams have not started.
Licensing the database and dropping a lookup into your existing navigation hook is a days-to-weeks integration. Building an equivalent in-house is a multi-quarter data-engineering project with an ongoing headcount line.
The specific failure mode differs slightly by product category, but the underlying gap — no external, deterministic check on the URL a model is about to touch — is identical across all three, which is why the same database works for all three.
Your agent drives a real or virtual browser session. Every click resolves to a URL before the DOM interaction happens, which means a page-type check can run before the click renders — the cheapest possible place to intercept a bad navigation.
The model sees a screenshot and reasons about pixels, not URLs. The practical answer is to resolve the address bar or the frame’s current URL at each step and check it the same way — screenshot-world does not remove the need for a URL-level gate, it just moves where you read the URL from.
Teams migrating from selector-based RPA to agentic execution lose the implicit safety of hand-built selectors that only ever pointed at known-good elements. A page-type check restores an equivalent boundary without hand-authoring it per customer.
A concrete example: a browser-agent vendor selling to procurement teams lets its agent research vendors across the open web — reading pricing, product, and documentation pages freely. Without a page-type check, the same agent that researches Stripe’s pricing page can, on a bad day, end up on Stripe’s login page or a checkout flow if a task description is ambiguous or a page injects a misleading link. With the database embedded at the navigation hook, that same agent gets a page-type answer — pricing, allow; login, deny — before either request leaves your infrastructure, on a domain your team never had to hand-classify.
Agent platforms already have a navigation hook — the point where the framework decides to actually issue a browser action or an HTTP fetch on the model’s behalf. The page-type check is one lookup inserted at that exact point, before the request leaves your infrastructure.
Your existing tool-call or browse-action layer.
Local index hit (licensed database) or one API call.
page_type, IAB, filtering category matched to your rules.
Denied requests return a reason the model can reason about.
Every decision is the audit trail your buyer’s security team asked for.
The pseudocode above is intentionally framework-agnostic: the same lookup slots into a LangChain tool wrapper, an OpenAI Agents SDK guardrail hook, a Playwright route interceptor, or a custom browser-use tool. What changes between frameworks is where the hook lives, not what the data returns.
Two starting points, both available as a redistribution license for your own customer base. Figures below are the standing self-serve and one-time prices; OEM redistribution terms are negotiated per platform and are not listed publicly.
| Path | What it covers | Standing price | Best for |
|---|---|---|---|
| Lookup API | Per-URL JSON checks, all 28 page types + both taxonomies | $99–$1,997/mo by volume | Prototyping, low-volume tiers, fastest integration |
| Database — 10M domains | Full schema, on-prem, perpetual | $14,999 one-time | Platforms wanting zero per-call cost and no external dependency |
| Database — 15M domains | Full schema, on-prem, perpetual | $24,999 one-time | Broader long-tail coverage for global customer bases |
| Database — 30M domains | Full schema, on-prem, perpetual | $49,999 one-time | Platforms that cannot afford an “unknown domain” answer |
| Monthly refresh (any tier) | Re-verified URLs, new domains, taxonomy updates | 30% of license price / yr | Any production deployment — staleness is a security property here |
| OEM redistribution | Embed inside your product for your own customers | Custom — contact us | Agent platforms shipping the guarantee as a feature |
40M+ full-repository cuts and volumes beyond the tiers above are quoted on request. Full self-serve pricing lives on the pricing page; this table exists so your procurement team does not have to hunt for it.
One practical note for OEM evaluations: most platform vendors prototype against the lookup API first, because it requires no data engineering to try — point your staging navigation hook at one endpoint and you have working page-type checks the same day. The decision to move to an on-prem database license usually comes later, once volume or latency requirements make a local index the better economics, and once the OEM redistribution terms are settled for your specific customer base and deployment footprint.
Enterprise buyers evaluating an agent platform in 2026 tend to ask some version of the same handful of questions. Having a concrete, mechanism-level answer to each is the difference between a stalled deal and a closed one.
Platform teams that have already shipped this tend to converge on the same rollout shape, whether they started on the API or went straight to an on-prem license.
Wire the lookup into the navigation hook in log-only mode first: record what would have been allowed, denied, or flagged without changing behavior. This surfaces false positives against your own customers’ real traffic before anything is enforced.
Turn on hard denial for the identity and commerce page types — login, signup, password_reset, checkout, cart — plus content-write types, since these carry the least ambiguity and the highest downside if missed.
Page types like upload and subscribe, and any customer-specific approved-vendor exceptions for checkout, get a human-in-the-loop flag rather than a hard deny, refined against real usage until the false-positive rate is acceptable to your customers.
If your platform sits behind an AI gateway or is embedded inside a larger enterprise browser deployment, the buyer conversation changes slightly — see how those teams frame the same data on AI gateway URL policy and enterprise browser agent control. For the full mechanism this page assumes, start with agent guardrails and the page-types database overview. Outside agent-specific guardrails, Alpha Quantum also runs AI Tools Blocklist, which blocks human employees from reaching unapproved AI tools — a policy layer some platform customers ask for alongside agent controls.
License the page-type map, egress rules, and host list under your own brand. Start with the sample, then talk OEM terms.
