Shopping and price-comparison agents already browse product catalogs on behalf of shoppers, and that traffic is only growing. Most of it is exactly what a merchant wants — an agent reading your product and pricing pages and citing you back to a buyer. The risk sits at the pages your storefront was never built for an anonymous automated visitor to reach: account login, checkout, cart, and the newsletter subscribe form a coupon-scraping agent will happily fill out a thousand times. AI Agent Allowlist is the page-type database that responsible agent platforms and gateways already check before every navigation — so getting your own storefront correctly classified in it is the single highest-leverage thing a merchant can do to steer that traffic where it belongs.
Think of it as the difference between hoping every agent behaves and knowing that the ones respecting default-deny policy already have the right answer for your domain, without your team building or operating any enforcement of its own.
Several high-profile AI agent incidents in 2026 involved agents that escaped their intended scope and reached account, login and write surfaces they had no business touching — a Hugging Face breach, a hijacked wiki, an artifact-registry covert channel, and takeovers of four third-party accounts. A storefront is exactly the kind of site a wandering or manipulated agent can land on next.
would your agents have been stopped? check the incident analysisThis is not a product you install on your storefront, and it does not inspect your incoming traffic. It is the reference data layer that agent platforms, AI shopping assistants and enterprise browsers already query before they let an agent navigate anywhere. A merchant’s leverage over that system is indirect but real: how your own domain is classified in the database determines whether every agent that respects it treats your product and pricing pages as fair game and your checkout, account and subscribe pages as off-limits.
That indirection is worth being upfront about, because it is easy to overstate what a data layer can do. A merchant cannot force every AI system on the internet to check this database before visiting their site, any more than a robots.txt file can force every crawler to respect it. What has changed since 2026 is the incentive on the other side: agent platforms and gateways now have their own reasons — liability, enterprise-buyer requirements, plain reputation risk — to enforce default-deny policy using exactly this kind of page-type data, and a merchant's job is simply to make sure their own storefront gives that policy the right answer when it asks.
Passively, through the ecosystem. Agent gateways built on default-deny policy already refuse checkout, cart, login and subscribe page types on any of the 40M+ domains in the map — including yours, once your storefront is correctly classified. You do nothing per-agent; you benefit from every compliant agent operator’s own guardrail.
Actively, if you run your own agents. Merchants increasingly run agents of their own — competitor price-monitoring, catalog enrichment against supplier sites, customer-support bots that browse partner documentation. Those agents are outbound traffic on someone else’s site, and the same page-type map is what keeps them off logins and checkouts elsewhere, exactly as it would for any other buyer of the data. If your own team is the one wiring that outbound policy into a browse tool, see the LLM app developer integration pattern.
Most merchants only ever notice the second use case, because it involves code their own team writes. The first one is easy to miss entirely, since it requires no integration at all — only that the classification behind the scenes is accurate. That asymmetry is exactly why this page exists: the highest-leverage action for most storefronts is the one with no engineering ticket attached to it.
The page types below are the ones almost every storefront serves. Getting them right in the source data — the URLs your own site links to for each — is what separates “agents find my products” from “agents wander into my checkout.” None of these classifications come from a merchant filling out a form; they come from how the site itself links its own pages, which is also why keeping that structure clean pays off automatically at the next refresh cycle rather than requiring a support ticket.
The page type shopping agents exist to read: description, price, availability, reviews. This is the traffic you want.
allow: productSubscription boxes, membership tiers, bulk pricing — comparison agents cite these directly when a shopper asks “who’s cheapest.”
allow: pricingShipping policy, return policy, FAQ and contact pages answer the questions a shopping agent needs before it recommends you.
allow: about, help_center, legal, contactWhere a purchase is actually committed. Denied by default for any agent that has not been explicitly authorized for agent-initiated purchase under a human-approval flow.
deny: cart, checkoutCredential surfaces. An agent that reaches these is either testing stolen credentials or has drifted far outside a shopping task — neither should be allowed to proceed.
deny: login, signup, password_resetNewsletter signup and product review submission are the two page types most exposed to automated coupon and discount-code abuse when left open to any visiting agent.
deny: subscribe, commentThe difference is not visible in your analytics as a blocked-request count — it shows up as which of your pages an agent is allowed to reach in the first place, decided before any request is sent.
An unclassified or stale storefront record leaves an agent operator’s policy engine guessing. Some guessed paths resolve to the wrong page type entirely; others fall back to default-deny for the whole domain, and a merchant loses the citation traffic they wanted from comparison and shopping agents in the first place. A freshly verified record removes the guesswork on both sides.
This matters more the faster your catalog changes. A seasonal storefront that reorganizes its URL structure twice a year, or a marketplace vendor whose product pages move between templates, is exactly the profile where an unrefreshed page-type record drifts out of date fastest — and drift shows up as a missed sale or a wrongly denied checkout, not as an error message anyone sees.
There is also a category question underneath the page-type question. Every domain in the database carries an IAB content category and a filtering category alongside its page types, so an agent’s policy can say something more specific than “allow product pages everywhere” — a procurement agent’s policy might allow product and pricing pages only on domains classified in an approved retail vertical, and treat everything else as flag-for-review regardless of page type. Getting your storefront’s vertical classification right is a second, less obvious lever alongside page types: a home-goods retailer misclassified into an unrelated or higher-risk vertical can find itself denied by a cautious agent policy even on its perfectly ordinary product pages.
“Steer agents to product pages, gate checkout” sounds like one rule, but it plays out differently depending on how a storefront is built and who controls its URL structure. The rule itself never changes; what changes is who is responsible for making sure the underlying data reflects it.
A merchant on its own domain controls its full URL structure, which means it also fully controls the signal a page-type classifier reads. A clean, consistent link structure between product, pricing and policy pages is the single best thing an independent store can do to get correctly classified and stay that way through redesigns.
A vendor selling through a larger marketplace does not control the marketplace's own page-type classification, but benefits from it automatically — the marketplace platform's login, cart and checkout pages are already covered, and a vendor's individual product listings inherit that protection without any action on the vendor's part.
A retailer running several brand sites, regional domains and a separate payments subdomain has the most surface area to get right: each domain needs its own accurate record, and a checkout hosted on a third-party payment processor's domain needs to be covered by that processor's classification, not the retailer's own.
A retailer running its own price-monitoring agent against competitor catalogs, or a catalog-enrichment agent pulling spec sheets from supplier sites, is on the other side of exactly the policy this page describes. The same default-deny posture applies, just outbound instead of inbound.
This distinction — inbound classification versus outbound policy — matters because merchants of any size tend to be both at once. The same commerce team that wants shopping agents routed correctly onto its own product pages is often also standing up a price-monitoring or assortment-research agent internally, and it is easy to assume the two problems are unrelated. They are not: both are answered by the identical page-type map, read in different directions. Getting your own storefront classified is a data-quality task with no code to write; wiring your outbound agent's navigation through a policy check is an integration task, and the two are worth planning together rather than as separate initiatives handled by separate teams.
| Approach | What it catches | Where it breaks |
|---|---|---|
| Rely on generic bot-detection / WAF rules | Known scraper signatures, high request-rate patterns | A well-behaved shopping agent looks like a normal browser session; nothing distinguishes “fine on product pages” from “fine on checkout” |
| Block all non-human traffic to checkout by IP or user-agent | Obvious automation, until the next user-agent string | Breaks legitimate agent-assisted shoppers your policy actually wants to allow onto product pages |
| Correctly classified page types in a database agent platforms check | Any compliant agent, on any of 40M+ domains, without per-request detection | Requires the agent operator to enforce default-deny — which is precisely the incentive every serious operator now has after 2026 |
Full database licenses are one-time: $14,999 for the top 10M domains, $24,999 for 15M, $49,999 for 30M, with an optional refresh at 30% of the license price per year. Agent platforms and gateways sizing a purchase to include the long tail of storefronts — not just the largest retailers — typically look at the 30M tier; see the pricing page for current terms. Merchants managing which AI tools their own staff can access, separate from the agent-browsing question this page covers, typically look at our sibling product aitoolsblocklist.com.
Check your own domain against the free sample, then see full-tier and OEM options for agent platforms sizing coverage.
