AI Agent Allowlist
Home Page-Types Database Agent Guardrails 2026 Incidents API Docs Pricing
Resources
Use Cases (15) Industries & Buyers (12) Learn: Core Concepts (12) Implementation Guides (15) Comparisons (8) Agent Security Guides (22) Market & Frameworks (9) Schema & Data Reference (6) FAQ Glossary
Why It Matters
2026 Agent Incidents Category Targeting Database Refreshes Contact Customer Login
Download Free Sample
lookup api · v1 reference

API Documentation

One authenticated GET endpoint, two modes. Send the exact URL your agent is about to open and get an allow/deny verdict for it, evaluated against the database, the 40 egress rules, and the host list — or send a domain and get its verified page-type map for local policy. This page is the complete reference: authentication, parameters, response schema, errors, quotas, and runnable examples.

GET https://www.aiagentallowlist.com/api/check?url={domain-or-url}

Why this lookup exists: see how, in our own replay of the public record, our database + egress rules deny the web-facing steps of almost all recent AI agent incidents — including the Hugging Face incident.

The 2026 agent incidents, replayed
Quick start

Your first verdict in one line

Subscribe to any plan on the pricing page, copy the API key from your account area, and run this. That is the whole integration surface — there are no SDKs to install and no other endpoints to learn.

curl
# May my agent open this exact URL? (url= keeps the full URL — nothing stripped)
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=https%3A%2F%2Fstripe.com%2Flogin"
#  → "verdict": "deny", "matched": {"layer": "rules", "id": "login", ...}

# A bare domain works too: verdict at the domain root + the full page-type map:
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=stripe.com"

The response is a single JSON object: whether the domain is in the database, its primary language, the page-type map with one verified URL per confirmed page type, and how many lookups remain on your plan this cycle. Everything below is detail on that one request and its one response.

URL verdicts

Send a full URL, get an allow/deny verdict

The product has two data layers, and the API evaluates both. Layer one is the database: 40M+ domains, each with verified URLs for up to 28 page types — that is the page-type map every response carries. Layer two is the bundled policy files: 40 method-aware URL-pattern rules (the Egress Rules Library) and the ~150-entry High-Value Host List. Database licenses ship both files for local evaluation inside your own proxy; the API also evaluates them server-side, so one request answers the question your policy engine actually asks: may the agent open this exact URL?

curl · url verdict
# The DseWiki incident's edit URL — denied by the wiki_edit pattern rule,
# on any domain, whether or not the database has a row for it:
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=https%3A%2F%2Fwww.wikiservice.at%2Fdse%2Fwiki.cgi%3Faction%3Dedit%26id%3DSandKasten"

#  → "verdict": "deny", "matched": {"layer": "rules", "id": "wiki_edit", ...}

# The Hugging Face incident's entry vector (dataset upload) — denied by the database's verified upload URL:
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=https%3A%2F%2Fhuggingface.co%2Fnew-dataset"

#  → "verdict": "deny", "matched": {"layer": "page_type_db", "id": "upload", ...}

With url= (optionally method=, default GET) the response gains three fields — verdict (allow, deny, or flag), matched (which layer fired: high_value_hosts, page_type_db, rules, or default, with the matching entry's id and note), and the echoed url/method — alongside the domain's full page-type map. Evaluation order: a hard deny on the host list wins; then an exact match against the domain's verified page-type URLs; then the 40 pattern rules; unmatched reads pass and unmatched writes are denied by default. Rules and host entries marked write-only apply to write methods only (a GET of a package page or a public paste passes), a rule does not fire on help or documentation pages about its own topic, and when one URL carries both a read and an action label, the action label decides. Subdomains fall back to their base domain's row (chat.openai.com resolves to openai.com). One verdict costs one lookup. A bare-domain input runs the identical evaluation at the domain root (verdict_scope: "domain_root") — there is no mode that skips layers.

Authentication

API key, two ways to send it

Every plan includes an API key, issued when your subscription is activated — you will find it in your account area the moment payment completes. Send it with each request either as the X-API-Key header or as an api_key query parameter; both are accepted on every request.

curl · both methods
# Recommended: header authentication
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=stripe.com"

# Alternative: query-parameter authentication
curl "https://www.aiagentallowlist.com/api/check?url=stripe.com&api_key=YOUR_API_KEY"
  Keep the key out of logs: prefer the X-API-Key header in production. Query strings — and any keys embedded in them — routinely end up in proxy logs, web-server access logs, and browser histories. The query-parameter form exists for quick tests and environments where setting headers is awkward.

A request with no key, or with a key that does not match an account, returns 401. A valid key on an account that is not yet activated — or whose monthly quota is exhausted — returns 403 with an explanatory message. Details in the error reference below.

Request

Request parameters

The endpoint is GET https://www.aiagentallowlist.com/api/check over HTTPS. One input parameter, one method parameter, one key. Every request runs the full three-layer evaluation — there is no reduced mode.

ParameterRequiredDescription
url required What to evaluate: a full URL or a bare domain. Nothing is ever stripped. A full URL gets its verdict for that exact URL (verdict_scope: "url"); a bare domain is evaluated at its root (verdict_scope: "domain_root") and returns the domain’s full page-type map either way. A value that cannot be parsed into a host returns 400. (domain is accepted as a legacy alias with identical behavior — it does not strip anything either.)
method optional The HTTP method the agent intends to use. Default GET. Matters at the default layer: unmatched reads pass, unmatched writes (POST, PUT, …) are denied.
api_key optional Your API key, if you are not sending it as the X-API-Key header. One of the two must be present on every request.
  Remember to URL-encode the url value. Percent-encode it (encodeURIComponent, urlencode, or your HTTP client’s params mechanism) so the checked URL’s own query string does not get mixed into the API request’s. wiki.cgi?action=edit&id=X sent unencoded would split at the & and the rule layer would only see half the URL. All four code samples below encode correctly.
Response

Response schema

Successful lookups return 200 with a JSON object. The shape is identical whether the domain is found or not — only found and the contents of page_types change — so one parser handles both cases.

FieldTypeMeaning
domainstringThe normalized domain the lookup resolved to. With url=, subdomains fall back to their base domain’s row when the full host has no row of its own (chat.openai.com → openai.com; the queried host is then echoed in subdomain_queried). Always echo-check this against what you sent.
verdictstringAlways present. allow, deny, or flag — the full three-layer evaluation runs on every request. For a full-URL input it applies to that exact URL and method; for a bare-domain input it applies to the domain root (see verdict_scope).
matchedobjectAlways present. Which layer produced the verdict — layer is high_value_hosts, page_type_db, rules, or default — with the matching entry’s id (e.g. wiki_edit, upload), its page_type, and a human-readable note. Log this next to every denied request and your audit trail explains itself.
verdict_scopestringurl when you sent a full URL; domain_root when you sent a bare domain and the verdict was evaluated at its root. A domain_root verdict is not a judgment about any deeper path — for a specific page, send that page’s URL.
url, methodstringThe evaluated URL and method, echoed back verbatim (for a bare-domain input, the evaluated URL is the domain root).
foundbooleantrue if the domain is in the classified database; false if it is not. For a guardrail, false is itself a signal — see the not-found example below.
languagestringDetected primary language of the site as an ISO 639-1 code (e.g. en). Present on found domains.
page_typesobjectMap of page-type name → verified URL on the domain. Keys are drawn from the 28 page types listed below. Only types that actually exist on the domain appear — if a key is absent, the classification pipeline confirmed there is no such page. Empty object ({}) when found is false.
iab_categorystring | nullIAB content category of the domain (e.g. News and Politics > National News), from our 700+-category taxonomy — the vertical axis for policies like “allow docs pages on Software-vertical domains only.” null where the domain has no IAB classification yet; coverage is strongest on higher-traffic domains and grows with each refresh cycle.
filtering_categoriesarrayWeb-filtering categories of the domain from the 59-category risk taxonomy (e.g. ["Financial Services","Online Banking"]) — the risk axis for deny-everywhere classes such as gambling or adult. Empty array where the domain is not yet in the filtering set.
open_page_ranknumber | nullPopularity and link-authority score of the domain on the 0 to 10 Open PageRank scale (e.g. 6.76 for stripe.com, 8.59 for github.com) — a trust prior for policies such as “let agents transact only on established domains” or “flag login pages on unknown low-authority hosts”. null where no score is available.
global_rankinteger | nullGlobal popularity rank position on the same Open PageRank scale (1 = most linked domain on the web). Present for roughly the top 40 million ranked domains; null otherwise.
quota_remainingintegerLookups left on your plan in the current 30-day cycle, after this request. Returned on every metered response, found or not, so you never need a separate usage call.

The 28 page types

page_types keys are drawn from a fixed vocabulary of 28 page types — the full set:

login signup password_reset checkout post_create comment cart upload subscribe product community events pricing documentation contact about leadership careers partners blog status legal case_studies security help_center integrations sitemap press

The red types — login, signup, password_reset, checkout, post_create, comment — are the credential, transaction, and write surfaces most agent policies deny by default; the amber types — cart, upload, subscribe, product, community, events — are the flag-for-review surfaces; the rest are research-safe allows. As everywhere in the schema, a key is present only when the page was verified to exist on the domain. The eight action types — signup, password_reset, cart, checkout, upload, post_create, comment, subscribe — mark pages where an agent acts (creates an account, submits a form, spends money, publishes content) rather than reads; they follow the same verified-only rule and are rolling out with the current reclassification cycle, so records gain them as their domains are re-verified. The full page-type definitions are on the database page.

Domain found

Each key maps to the verified live URL — discovered by traversing the site’s actual link structure, not guessed from path patterns.

200 · json
{
  "domain": "github.com",
  "found": true,
  "language": "en",
  "page_types": {
    "login": "https://github.com/login",
    "signup": "https://github.com/signup",
    "pricing": "https://github.com/pricing",
    "documentation": "https://docs.github.com",
    "product": "https://github.com/features",
    "about": "https://github.com/about",
    "careers": "https://github.com/about/careers",
    "contact": "https://support.github.com",
    "legal": "https://docs.github.com/site-policy/github-terms/github-terms-of-service",
    "security": "https://github.com/trust-center",
    "community": "https://github.com/orgs/community/discussions",
    "case_studies": "https://github.com/customer-stories",
    "partners": "https://github.com/partners",
    "integrations": "https://docs.github.com/integrations/concepts/about-building-integrations",
    "events": "https://github.com/resources/events",
    "blog": "https://github.com/resources/articles",
    "press": "https://github.com/newsroom",
    "sitemap": "https://github.com/sitemap",
    "subscribe": "https://github.com/newsletter"
  },
  "iab_category": "Technology & Computing > Computing > Programming Languages",
  "filtering_categories": ["CDN / Hosting / Infrastructure", "Developer Tools / Code Hosting", "Freeware/Software", "Web-based Applications"],
  "open_page_rank": 8.59,
  "global_rank": 22,
  "quota_remaining": 89999
}

This is the real record for github.com from the free sample database — every URL above was manually verified live.

Domain not found

Still 200, still one credit — and for a guardrail, a verdict in its own right: an unclassified domain is a default-deny candidate.

200 · json
{
  "domain": "example-unknown.com",
  "found": false,
  "page_types": {},
  "quota_remaining": 89998
}
  Absence is information. Two distinct negatives matter to policy code: a type missing from page_types on a found domain means the pipeline confirmed the domain has no such page; a domain returning found: false means the classification layer has never seen it. Treat the first as “this surface does not exist here” and the second as “this destination is unvetted” — under a default-deny posture, the second should block or escalate.
Errors

Error reference

Errors use conventional HTTP status codes with a JSON body carrying a human-readable message. Quota-related 403s also include quota fields so your client can distinguish “not activated” from “used up”.

CodeMeaningWhat your client should do
200SuccessParse the JSON. Check found before reading page_types.
400Invalid domain parameterThe value was missing or could not be parsed into a domain. Fix the input; do not retry unchanged.
401Missing or invalid API keyNo key was sent, or the key matches no account. Verify the key against your account area; do not retry until corrected.
403Account not activated, or monthly quota exhaustedRead the JSON message and quota fields to tell the cases apart. Not activated: complete payment/activation. Quota exhausted: wait for the next 30-day cycle or upgrade — there is no automatic overage billing.
429Too many requestsYou hit the per-IP burst guard (roughly 240 requests/minute, independent of your quota). Back off briefly and retry; smooth bursts with a local queue.
503Backend temporarily unavailableTransient. Retry with exponential backoff. In a guardrail, fail closed or serve from your local cache rather than skipping the check.
  Guardrail failure posture: decide up front what your enforcement point does on 429/503. The safe default is to fail closed for unknown domains and fall back to your local cache for domains you have already resolved this cycle — an outage should never widen agent access.
Code samples

The same lookup in four languages

Each sample reads the key from an environment variable, sends it as a header, passes the domain safely encoded, and handles the found / not-found / error cases. Copy, set AAL_API_KEY, run.

curl
# Bare domain
curl -H "X-API-Key: $AAL_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=stripe.com"

# Full URL — let curl handle the encoding with --data-urlencode + -G
curl -G -H "X-API-Key: $AAL_API_KEY" \
     --data-urlencode "url=https://dashboard.stripe.com/login" \
     "https://www.aiagentallowlist.com/api/check"
python · requests
import os
import requests

API_URL = "https://www.aiagentallowlist.com/api/check"
HEADERS = {"X-API-Key": os.environ["AAL_API_KEY"]}

def check_domain(domain):
    # `domain` may be a bare domain or a full URL — the API normalizes it
    resp = requests.get(API_URL, headers=HEADERS,
                        params={"domain": domain}, timeout=10)
    if resp.status_code == 429 or resp.status_code == 503:
        raise RuntimeError(f"Transient error {resp.status_code} — retry with backoff")
    resp.raise_for_status()
    return resp.json()

record = check_domain("https://dashboard.stripe.com/login")
if record["found"]:
    login_url = record["page_types"].get("login")
    print(f"login page: {login_url}, quota left: {record['quota_remaining']}")
else:
    print("unclassified domain — default-deny candidate")
node · fetch
const API_URL = "https://www.aiagentallowlist.com/api/check";

async function checkUrl(target) {
  const url = `${API_URL}?url=${encodeURIComponent(target)}`;
  const res = await fetch(url, {
    headers: { "X-API-Key": process.env.AAL_API_KEY }
  });
  if (res.status === 429 || res.status === 503) {
    throw new Error(`Transient error ${res.status} — retry with backoff`);
  }
  if (!res.ok) throw new Error(`API error ${res.status}`);
  return res.json();
}

const record = await checkDomain("https://dashboard.stripe.com/login");
if (record.found) {
  console.log("login page:", record.page_types.login ?? "none on this domain");
  console.log("quota left:", record.quota_remaining);
} else {
  console.log("unclassified domain — default-deny candidate");
}
php · curl
<?php
function check_domain($domain) {
    $url = 'https://www.aiagentallowlist.com/api/check?url=' . urlencode($domain);
    $ch  = curl_init($url);
    curl_setopt_array($ch, [
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_TIMEOUT        => 10,
        CURLOPT_HTTPHEADER     => ['X-API-Key: ' . getenv('AAL_API_KEY')],
    ]);
    $body   = curl_exec($ch);
    $status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);
    curl_close($ch);

    if ($status === 429 || $status === 503) {
        throw new RuntimeException("Transient error $status — retry with backoff");
    }
    if ($status !== 200) {
        throw new RuntimeException("API error $status: $body");
    }
    return json_decode($body, true);
}

$record = check_domain('https://dashboard.stripe.com/login');
if ($record['found']) {
    echo 'login page: ', $record['page_types']['login'] ?? 'none on this domain', PHP_EOL;
    echo 'quota left: ', $record['quota_remaining'], PHP_EOL;
} else {
    echo 'unclassified domain — default-deny candidate', PHP_EOL;
}
Integration pattern

Using the API in an egress guardrail

Two patterns, by volume. The simple one: your proxy sends every intended navigation as url= (plus method=) and enforces the returned verdict — all three layers evaluated server-side, one call, nothing to implement. The high-volume one, below: cache each domain’s page-type record locally and match URLs in your own hook — agents revisit the same domains constantly, so one cached lookup covers every URL on that domain. If you run the cached pattern, deploy the two bundled policy files (rules + host list) locally as well, or you are enforcing only the database layer.

pseudo-code · egress hook
# Policy: which page types this fleet may touch
ALLOW_TYPES = {"pricing", "documentation", "blog", "about", "contact"}
DENY_TYPES  = {"login", "signup", "password_reset", "checkout", "post_create", "comment"}

def evaluate_navigation(url):
    domain = extract_domain(url)          # extract yourself — never send full URLs as `domain`

    record = cache.get(domain)            # local cache first — one lookup per domain
    if record is None:
        record = api_check(domain)        # GET /api/check?url=<domain> — cache the map
        cache.set(domain, record, ttl=CACHE_TTL)

    if not record["found"]:
        return DENY("unclassified domain")   # default-deny: unknown = unvetted

    page_type = match_url_to_type(url, record["page_types"])
    if page_type in DENY_TYPES:
        return DENY(f"denied page type: {page_type}")   # log + alert
    if page_type in ALLOW_TYPES:
        return ALLOW(page_type)
    return FLAG(url)                      # unmatched URL on a known domain → review

Three practices make this pattern hold up in production. Cache aggressively: keying the cache by domain means a fleet that opens ten thousand URLs a day on five hundred domains spends five hundred lookups, not ten thousand. Log every verdict — allow, deny, and flag alike — with the page type and URL, so denials feed alerts and allows feed policy reviews. Keep the default-deny branch honest: the found: false path is where the 2026-style incidents get stopped, so route it to denial or human review, never to silent allow. Broader policy design — fleet scoping, chokepoints, audit trails — is covered in the enterprise guardrails guide and on the agent guardrails page.

The other two layers: the Egress Rules Library (40 method-aware URL-pattern rules, including payment and admin surfaces, which are handled there rather than as page-type keys) and the High-Value Host List. With url= the API evaluates both server-side on every verdict; database licenses ship both files so the cached/local pattern can enforce them without any API call.

Quotas & plans

Metering, rate limits, and plan quotas

Each successfully served lookup — found or not — consumes one credit. Quotas run on 30-day cycles from your subscription date, and every metered response carries quota_remaining. Separately from the quota, a coarse per-IP burst guard of roughly 240 requests per minute smooths traffic spikes; hitting it returns 429 and does not consume credits.

PlanPrice / monthLookups / month 
Advanced$499450,000Subscribe →
Advanced Plus$999900,000Subscribe →
Business$1,9972,750,000Subscribe →
Enterprisefrom $30,000/yrcustom, 10M+Talk to us →

Exhausted quotas return 403 until the next cycle — there is no automatic overage billing and your card is never charged beyond the plan price. Need more than 10M lookups a month, or want the data on-prem with no metering at all? See database licenses and OEM options on the pricing page.

FAQ

API questions, answered

What does one lookup return?
Every response carries both halves: an allow/deny/flag verdict with the matched layer and rule that produced it (for the exact URL you sent, or for the domain root if you sent a bare domain), plus the domain’s full record — found, its detected primary language, and a page_types object mapping each confirmed page type (login, signup, checkout, cart, pricing, documentation, and the rest of the 28-type vocabulary) to its verified URL. Every metered response also includes quota_remaining.
How do I authenticate?
Send your API key either as an X-API-Key header (recommended for production — it keeps the key out of URL logs) or as an api_key query parameter. Keys are issued on subscription and shown in your account area the moment your plan is activated.
What does found: false mean for a guardrail?
The domain is not among the 40M+ classified domains. For an egress guardrail that is itself a policy signal: a destination the classification layer has never seen is unvetted, and under a default-deny posture it should be denied or routed to human review rather than silently allowed. The not-found response still returns 200 and consumes one credit.
A page type is missing from page_types — did the check fail?
No. Only page types that actually exist on the domain appear in the map. An absent key on a found: true response means the classification pipeline confirmed the domain has no such page — a verified negative your policy can rely on, not a coverage gap.
Can I pass a full URL instead of a bare domain?
Yes — url= takes either, and nothing is ever stripped. A full URL gets its verdict for that exact URL; a bare domain gets the identical evaluation at the domain root. (The legacy domain parameter behaves exactly the same — it is just an alias.) URL-encode the value so the checked URL’s own query string is not mixed into the API request’s.
How are lookups counted, and what happens at the quota?
Each successfully served lookup consumes one credit, found or not. All plans are additionally rate-limited to 300 lookups per minute; bulk replication of the database via the API is prohibited under fair use (database licenses cover bulk needs). Quotas run on 30-day cycles from your subscription date — 450,000/month on the $499 Advanced plan up to custom enterprise volumes. Beyond the quota, requests return 403 with a message and quota fields until the next cycle; there is no overage billing. Upgrading applies the larger quota immediately.
Is there a rate limit separate from the quota?
Yes — a coarse per-IP burst guard of roughly 240 requests per minute, independent of your monthly quota. Hitting it returns 429; back off briefly and retry. A local queue plus a per-domain cache keeps almost every real deployment well under it.
Should I cache responses?
Yes, and by domain. Page-type maps change on the timescale of site redesigns, not requests, so caching a domain’s record locally is both allowed and recommended — it cuts your credit usage dramatically and keeps your guardrail’s decision latency independent of the network. On 429/503 errors, serving from the cache while failing closed for unknown domains is the recommended posture.

Get a key and make your first lookup today

Plans start at $99/month for 90,000 lookups. Subscribe, pay by PayPal or card, and your key is active the moment payment completes.

View Plans & Subscribe