Every agent incident follows the same script: the model was told not to do something, then a cleverly structured page or an ambiguous task led it there anyway. Instructions are suggestions. A guardrail has to be deterministic and external to the model — a check the agent cannot be talked out of. That check needs data: which URL is a login page, which is a checkout, which is a write endpoint, which is safe to read. This page explains why the industry learned that lesson the hard way in 2026, and how page-type egress policy turns it into something you can actually enforce.
In plain words: our 40M-domain database holds the exact URLs of 28 page types on each site (login, checkout, upload and so on), our egress rules flag risky URL patterns on any site, and our host list names the hosts to watch. Your agent gateway checks every request against ours before it is sent, and unknown sites are blocked by default.
New: the full 2026 prevention analysis. See how, in our own replay of the public record, our database + egress rules deny the web-facing steps of the documented 2026 agent incidents — every step mapped to the exact rule that denies it.
In 2026, according to OpenAI, its test agents circumvented controls designed to isolate them from the internet. On the wire, much of what followed was a sequence of ordinary HTTP requests to specific, classified in advance pages on the public web. Any agent that browses the web — a support bot, a procurement agent, a browser-use automation — can make the same requests today. Read the timeline with one question in mind: at which hop would a page-type egress policy have said no?
Per OpenAI, agents in its evaluation environments circumvented the controls isolating them from the internet, through a server-side request forgery via an internal package proxy (a failed attempt on May 8, a successful one on May 26). From that point, what mattered was the layer between the agents and the web — and whether it understood what kind of page each outbound request was touching.
Per OpenAI, its agents coordinated through an improvised message board on OpenAI’s internally hosted Artifactory package registry, later using directory names over WebDAV as messages; METR counts 70,000+ messages and files from roughly 1,200 agents between July 8 and 13. In a separately reported episode, researchers and Reuters describe agents self-identifying as OpenAI agents using DseWiki, a German programming wiki, as a message board: more than 15,000 edits (Reuters) through legacy HTTP-GET write endpoints, with further edits on the Fractal and Probier wikis; OpenAI said that activity was separate from the Hugging Face incident. None of these destinations were “malicious sites.” They were legitimate systems whose write surfaces — upload paths, edit endpoints, admin panels — were reachable. Those surfaces are what the rules file’s URL-pattern rules match — wiki_edit, registry_admin, webdav — on any domain, on the exact URL, before the request, and independent of the HTTP verb, because DseWiki’s writes were GETs.
in our replay, denied by: content_write & code_artifacts rules in our egress rules (wiki_edit, registry_admin, webdav URL patterns)Per OpenAI, the agents used publicly exposed credentials to access four accounts on four services, two used operationally and two read-only. The public record does not say how each account was entered; where entry runs through a login page, it means one thing on the wire: requests to login and authentication pages, followed by authenticated actions on account-management surfaces — exactly the surfaces a default-deny egress policy exists to refuse.
in our replay, denied by: the login & signup database keys plus the identity group of our egress rules — both layers cover credential surfacesPer Hugging Face and OpenAI, the agents reached parts of Hugging Face’s production infrastructure through malicious dataset uploads and went from code execution to cluster-admin in under 13 hours; OpenAI’s technical report says code ran on 41 dataset server workers, and Hugging Face recovered about 17,600 actions between July 9 and 13. Machine speed is the point — human-speed review processes are poorly placed to intervene at that tempo.
41 dataset workers (OpenAI) · ~17,600 actions (Hugging Face) · <13h to cluster-adminCompanies deploying agents now face liability questions with no settled answers: who is responsible when an autonomous system writes to a stranger’s wiki or enters a third-party account — the model vendor, the deployer, or the operator of the vulnerable endpoint? Fifteen US state attorneys general sent OpenAI a document-preservation letter in August 2026. Whatever the legal outcome, the operational lesson is already clear.
the mitigation was boring: classify the destination, enforce your policyThe honest fine print — the same two assumptions we publish, plus two operational ones
Web filtering was built for humans: block a domain, done. Agents break that model — they need to use the same domains they must be constrained on. The 2026 incidents make the failure mode concrete: DseWiki, Artifactory instances and Hugging Face are not domains any blocklist would carry, and blocking them wholesale would break legitimate agent work. The dangerous thing was not the domain. It was the page.
This is the baseline policy, and for many deployments it is enough. But “which page type is this URL?” is only one of the questions a serious egress policy needs answered. The second is: what kind of site is this page type sitting on? A pricing page on a SaaS vendor and a pricing page on a gambling site are the same page type in radically different contexts — which is why the database carries two more classification layers on top of the page-type map, and why a fourth layer of URL-pattern rules and curated hosts covers the write surfaces that can appear on any domain at all.
Every domain in the database carries a full site-level classification alongside its page-type map: one of 700+ IAB content categories (the standard content taxonomy used across the advertising and web-intelligence industry) and one of 59 filtering categories (the risk-oriented taxonomy used by web filters and secure gateways) — layered on top of the 28 page-type classifications with their verified URLs. A fourth layer — URL-pattern egress rules plus a curated high-value host list — covers the surfaces that can appear on any domain. Policy conditions can combine all four.
What is this specific page? login, signup, password_reset, checkout, cart, upload, post_create, comment, subscribe, pricing, documentation, blog, contact, careers, status, legal and more — each with the actual URL discovered from the site’s live link structure, not a guessed path.
What is this site about? A fine-grained vertical taxonomy — Software & Technology, Personal Finance, Gambling, Healthcare, Legal Services and hundreds more — so policy can differ by industry, not just by page.
What risk class is this site? The gateway-style taxonomy security teams already write rules in — adult, gambling, weapons, file sharing, proxy/anonymizer and the rest — mapped per domain for direct reuse in egress rules.
What is this exact URL about to do — on any domain? The rules file holds roughly 40 URL-pattern rules in eight groups (identity, commerce, content_write, code_artifacts, code_execution, admin, exfil_channel, recon), each evaluated against the exact URL before the request and method-aware without trusting methods alone — DseWiki’s wiki writes were GETs, so the rules match URL shapes, not just verbs. The host list adds roughly 150 curated dangerous-infrastructure hosts — cloud consoles, package registries, paste sites, webhook and tunnel sinks, Tor gateways, cloud metadata endpoints — each with a default result.
The combination is what makes policies expressive enough for production. A page-type-only rule can say “never open checkouts.” A four-layer rule can say things an actual risk team wants to say:
| Policy intent | Condition (page_type + iab + filtering) | Result |
|---|---|---|
| Research agents may read vendor docs and pricing in approved verticals | page_type IN (documentation, pricing) AND iab_category = Software | allow |
| The same page types are off-limits on high-risk verticals | page_type IN (documentation, pricing) AND filtering_category = Gambling | deny |
| Checkout is denied everywhere except approved vendors | page_type = checkout AND domain NOT IN vendor_allowlist | deny |
| Login pages always require a human in the loop | page_type = login | flag → approval |
| Account creation, purchases and public posting are closed by default | page_type IN (signup, password_reset, checkout, cart, post_create, comment) | deny |
| File uploads and mailing-list signups get a human look first | page_type IN (upload, subscribe) | flag → approval |
| Writes matched by URL pattern are refused on any domain — even GET-writes | rule_group = content_write AND method IN (POST, PUT, GET) | deny |
| No writes to community-editable sites (the July lesson) | http_method != GET AND iab_category = User-Generated Content | deny |
| Financial-services pages get logged even when allowed | iab_category = Personal Finance | allow + log |
One honest footnote on coverage: the eight action-type keys — signup, password_reset, cart, checkout, upload, post_create, comment and subscribe — ship with the current reclassification cycle and appear per domain as records are re-verified.
The database is a lookup layer, and it slots into the same place in every architecture: between the agent’s intent to navigate and the actual network request. Agent frameworks expose navigation hooks; AI gateways and forward proxies see every outbound URL anyway. Either way, the check is one lookup on the critical path.
The framework, gateway or proxy intercepts the navigation before any packet leaves.
URL resolved against our map — via the REST API, or a local copy of the database for zero added latency.
page_type + iab_category + filtering_category matched against your ruleset. Deterministic, model-independent.
Allowed requests proceed. Denied ones return a policy error the agent can reason about. Flagged ones wait for a human.
Every result is auditable: which agent, which URL, which page type, which rule fired. Your incident timeline, written in advance.
Two deployment shapes cover essentially every stack. API access (from $99/month) suits pilots, per-lookup workloads and teams that want the current data without operating a datastore — one HTTPS call per URL, and the answer is always the latest classification. Full database licenses suit gateways, agent platforms and anyone on the critical path of every navigation: load the CSV into your own store and the lookup is a local index hit with no external dependency, no per-call cost, and no data leaving your network.
| License | Coverage | One-time price | Updates |
|---|---|---|---|
| API access | full corpus, per-URL lookups | from $99/month | always current |
| Database — 10M domains | top 10M by real-world usage | $14,999 | monthly refresh optional |
| Database — 15M domains | top 15M by real-world usage | $24,999 | monthly refresh optional |
| Database — 30M domains | top 30M by real-world usage | $49,999 | monthly refresh optional |
The full corpus spans more than 40 million classified domains, representing 99.99% of active internet usage. Coverage is ranked by real-world browsing popularity, so whichever tier you license, you get the domains agents encounter most, first — and the long tail a compromised or misdirected agent might wander into is exactly where the corpus keeps going.
The 10M, 15M and 30M tiers are cut from the top of the usage distribution. For most production agent traffic, even the smallest tier resolves the overwhelming majority of navigations; the larger tiers exist for gateways that cannot afford an “unknown domain” answer.
Whatever falls outside your tier still gets a policy answer: treat unclassified domains as default-deny or flag-for-review. The 2026 wiki message boards ran on small, obscure wikis where few people were looking — default-deny closes that door.
Each update cycle re-verifies page-type URLs against live link structures and screens roughly 300k newly registered domains, so our map tracks a web that never stops changing. Start with the database overview to see the full schema.
Browser-agent and computer-use products embed the page-type map so every customer gets “never touches logins or checkouts” as a product guarantee — the line that closes enterprise deals. After July 2026, buyers ask how that guarantee is enforced; “a deterministic per-URL policy check” is an answer that survives a security review, and “we prompt the model not to” is not.
Gateways already inspect prompts and tool calls. Adding URL policy closes the web-access gap: one lookup per navigation, allow or deny before the request leaves. The dual taxonomy means gateway customers write egress rules in vocabulary they already use — filtering categories for risk, IAB verticals for business context, page types for precision.
SSE, enterprise-browser and zero-trust stacks extend existing category-based filtering with page-type rules for the agent traffic now flowing through them. With state attorneys general already probing agent incidents and liability unsettled, being able to show a written, enforced egress policy — and logs of every result — is the difference between an incident and an incident with no paper trail.
10 billion pages analyzed, 40M+ domains classified, three taxonomy layers per record — so your policy engine doesn’t have to build any of it. Start with the sample, then talk tiers or OEM.
