AI Agent Allowlist
Home Page-Types Database Agent Guardrails 2026 Incidents API Docs Pricing
Resources
Use Cases Industries & Buyers Learn: Core Concepts Implementation Guides Comparisons Schema & Data Reference FAQ Glossary
Why It Matters
2026 Agent Incidents Category Targeting Database Refreshes Contact Customer Login
Download Free Sample
reference · lookup api at a glance

The Page-Type Lookup API, Explained in Plain Terms

One endpoint, one request shape, one response shape. This page walks through what the API is, what it returns, how authentication and rate limits work, and what it costs — illustrated throughout on the entry-tier Pro plan ($99/month, 90,000 lookups) so every number below is one you can hold onto. For the exhaustive field-by-field reference, see the full API documentation; this page is the faster read that gets you oriented first.

0HTTP GET endpoint, no SDK
0Lookups/month on the $99 Pro plan
0Page types resolved per lookup
0Requests/min burst guard, every plan
The endpoint, in one line

GET /api/check?url={domain-or-url}

Every plan — Pro included — calls the exact same endpoint: GET https://www.aiagentallowlist.com/api/check. Send a domain or a full URL, get back a verdict and the domain's record. There are no separate endpoints for different page types, no batch mode, and nothing to install; the entire integration surface is one authenticated GET request.

curl · on the Pro plan
# Your Pro-plan key, 90,000 lookups/month, resolved with one call:
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=stripe.com"

# A full URL works identically — nothing is stripped:
curl -H "X-API-Key: YOUR_API_KEY" \
     "https://www.aiagentallowlist.com/api/check?url=https%3A%2F%2Fdashboard.stripe.com%2Flogin"
#  → "verdict": "deny", "matched": {"layer": "page_type_db", "id": "login", ...}

This mirrors the request shape documented in full on the API docs request reference exactly: one required url parameter (a bare domain or full URL, nothing ever stripped), an optional method parameter (default GET), and your key sent as a header or query parameter. Nothing here is a simplified or alternate surface — it is the same endpoint, described at a faster pace.

One accounting detail worth stating up front, since it shapes every number on this page: a lookup is counted per API call, not per field returned. A single request against the Pro plan's quota comes back with the full record — the page-type map, both content taxonomies, and popularity context — and that entire response still counts as exactly one of the plan's 90,000 monthly lookups. You are never billed per page type resolved or per category attached; the unit of billing is the call itself.

What comes back

The response, field by field

A successful call returns 200 and a single JSON object. The shape is identical whether the domain is found or not, and identical on every plan — a Pro-plan response and a Business-plan response for the same URL are byte-for-byte the same JSON, differing only in quota_remaining.

FieldMeaning
verdictAlways present. allow, deny, or flag for the exact URL and method sent, evaluated against the database, the ~40 egress rules, and the ~60-entry host list.
matchedWhich layer produced the verdict (high_value_hosts, page_type_db, rules, or default), with the matching entry's id and a human-readable note. Log this next to every deny.
foundWhether the domain is among the 40M+ classified domains. false is itself a policy signal — an unclassified domain is a default-deny candidate.
page_typesMap of page-type name → verified URL, drawn from the 28-type vocabulary. Only types confirmed to exist on the domain appear; absence is a verified negative.
iab_category, filtering_categoriesThe domain's IAB content category (700+-category taxonomy) and web-filtering categories (59-category taxonomy), included in every lookup at no extra cost.
open_page_rank, global_rankPopularity and link-authority context on the 0–10 Open PageRank scale, and the domain's global rank position where available.
quota_remainingLookups left in your current 30-day cycle after this request — on the Pro plan, this counts down from 90,000.

This is the identical field set documented on the full API docs response schema — nothing here is abbreviated or renamed. If you have already read that page, this table is a refresher; if you have not, it is everything you need to parse a response correctly on your first attempt.

Authentication

One key, two ways to send it

Subscribing to the Pro plan (or any plan) issues an API key the moment payment completes, visible immediately in your account area. Send it as the X-API-Key header (recommended, since query strings end up in logs) or as an api_key query parameter — both are accepted on every request, on every plan, with no difference in behavior between them.

curl · both auth methods
# Header — recommended for production
curl -H "X-API-Key: YOUR_API_KEY" "https://www.aiagentallowlist.com/api/check?url=stripe.com"

# Query parameter — fine for a quick test
curl "https://www.aiagentallowlist.com/api/check?url=stripe.com&api_key=YOUR_API_KEY"

A missing or unrecognized key returns 401. A valid key on a Pro-plan account that has exhausted its 90,000-lookup monthly quota, or one that has not completed activation, returns 403 with a message distinguishing the two cases. Full error reference on the API docs errors table.

Quota, on the Pro plan specifically

What 90,000 lookups a month actually looks like

Numbers are easier to reason about when they are not abstract. Here is the Pro plan's quota worked through in practical terms, since it is the plan most first integrations start on.

Pro plan · monthly quota90,000 lookups
≈ 2,950 lookups/day on average across a 30-day cycle · ≈ 123/hour if spread evenly · $1.10 per 1,000 lookups

In practice, real agent traffic is not spread evenly — a research agent might burn a few thousand lookups in a single batch run, then sit idle for hours. What matters against the Pro plan's ceiling is the monthly total, not the hourly rate: a fleet that caches by domain (as recommended in the integration pattern section of the full docs) can support a meaningfully larger number of URL checks than 90,000, because revisits to already-cached domains cost nothing further against quota.

Separately from the monthly quota, every plan — Pro included — is subject to a per-IP burst guard of roughly 240 requests per minute, which exists to smooth traffic spikes rather than to cap total usage. Hitting it returns 429 and does not consume a lookup from your quota; back off briefly and retry. A local queue in front of your integration keeps almost any real deployment comfortably under it.

Once the Pro plan's 90,000 lookups are used in a given cycle, further requests return 403 with a clear message until the next 30-day cycle begins — there is no automatic overage billing, and your card is never charged beyond $99 for staying on this tier. Upgrading to Pro Plus (225,000/month for $249) applies the larger quota immediately, without waiting for the next cycle.

Errors, briefly

The status codes you actually need to handle

CodeWhat it means on the Pro plan
200Success. Parse the JSON; check found before reading page_types.
400The url parameter was missing or unparseable. Fix the input before retrying.
401No API key sent, or the key does not match a Pro-plan (or any) account.
403Account not activated, or the Pro plan's 90,000-lookup quota is exhausted for this cycle.
429The ~240 requests/minute burst guard was hit. Back off and retry; no quota consumed.
503Transient backend issue. Retry with backoff; fail closed rather than skipping the check.

Every code here matches the error reference in the full API docs exactly. The one guardrail-specific decision worth making up front, regardless of plan: what your enforcement point does on 429 or 503. The safe default is to fail closed for domains you have not already resolved this cycle, and serve from your local cache for domains you have.

Two of these codes deserve a closer look because they are easy to confuse with each other during on-call debugging. A 403 on the Pro plan almost always means one of two things: the account has not finished activation, or the 90,000-lookup monthly quota is spent for the current cycle — the JSON body's message and quota fields distinguish the two, so a client should read that message rather than treating every 403 identically. A 429, by contrast, has nothing to do with the monthly quota at all; it is the per-IP burst guard reacting to request rate, and it resolves itself within the same minute once traffic slows, whereas a quota-exhaustion 403 persists until the next billing cycle or an upgrade. Conflating the two in an alerting system produces false alarms in one direction and missed ones in the other.

API vs. on-prem database

When the Pro plan is the right starting point, and when it isn't

The API and the on-prem database license return the same schema, so the choice is about deployment shape, not data quality.

ConsiderationPro plan API ($99/mo)Database license (from $14,999 one-time)
Setup timeMinutes — subscribe, get a key, call the endpointLoad a CSV/JSON/SQL delivery into your own store
Data freshnessAlways current — hits the live databaseFrozen at delivery date unless monthly refreshes are added
Latency per lookupOne network round-tripLocal read, no external dependency
Best fitPrototypes, small fleets, workloads under ~10M lookups/monthLatency-sensitive or data-residency-constrained production systems

Most integrations start on the Pro plan precisely because it removes every infrastructure decision except "call this URL." The full build-vs-buy comparison, including a self-hosted crawler as a third option, is worked through in detail in the implementation guide.

Making 90,000 lookups go further

Caching turns a monthly quota into an effective ceiling much higher than the number on the plan

The Pro plan's 90,000-lookup quota describes API calls, not URLs checked. Because agent traffic revisits the same domains constantly — a research agent looking at the same fifty vendor sites across dozens of tasks, a support agent hitting the same handful of documentation domains all day — a cache keyed by domain lets one lookup cover every URL an agent subsequently opens on that domain, for as long as the cache entry is kept.

Concretely: an agent fleet that opens ten thousand URLs a day across five hundred distinct domains does not need ten thousand lookups a day if it caches by domain — it needs, at most, five hundred, the first time each domain is seen in a given cache window. On the Pro plan's 90,000-lookup monthly allowance, that arithmetic is the difference between running out of quota by the second week of a billing cycle and running comfortably under it with headroom to spare. The pseudo-code pattern for this — check the local cache first, call the API only on a miss, cache the resulting record — is laid out in full in the integration pattern section of the full API docs.

The one thing caching should never do is cache a deny into a permanent allow, or vice versa, past the point where the underlying data might have changed. Page-type URLs move on the timescale of site redesigns, not requests, so a cache lifetime measured in hours to a few days is typical — long enough to capture nearly all the repeat-visit savings, short enough that a re-verified change in the database eventually supersedes a stale local entry rather than being shadowed by it indefinitely.

There is a second reason to cache beyond quota economics: latency. A cached record resolves in the time it takes to read local memory or a fast key-value store; an API call is a network round trip, typically single-digit to low double-digit milliseconds, but not free, and not something you want on the critical path of every single navigation decision an agent makes if the same domain was already resolved moments earlier. Treating the API as the source of truth for a domain's record, refreshed periodically, rather than as a per-URL gate called on every single fetch, is the pattern almost every production deployment converges on regardless of which plan they are on.

Growing past the Pro plan

When 90,000 lookups a month stops being enough

The Pro plan is sized for a single integration or a small agent fleet doing its own domain caching. Three signals tend to show up together when it is time to move to a larger tier, and none of them require a rewrite — every plan on this page shares the same endpoint, the same authentication, and the same response schema.

SignalWhat it looks like in practiceLikely next step
Quota exhausted mid-cycle403s appear before the 30-day cycle resets, even with domain caching in placePro Plus ($249/mo, 225,000/mo) or Advanced ($499/mo, 450,000/mo)
More agents, more domainsNew agent roles added to the fleet, each touching a distinct set of domains with little overlap to cache againstAdvanced Plus ($999/mo, 900,000/mo)
Gateway or platform-scale trafficThe lookup sits behind a shared proxy serving many downstream tenants or customersBusiness ($1,997/mo, 2,000,000/mo, $1.00/1,000) or a custom enterprise volume

Moving up a tier is a one-step change: subscribe to the larger plan and cancel the smaller one from your PayPal account, and the new quota applies immediately — no code changes, no new key format, no schema differences to account for. Full side-by-side numbers for every tier are on the pricing page.

How the Hugging Face breach could have been stopped

The 2026 Hugging Face breach involved an agent reaching a dataset-upload page it should never have touched. A single lookup on that URL — the same call shown throughout this page, on the same Pro plan — would have returned "verdict": "deny" matched against the upload page type, pre-request. Our analysis of the public disclosures shows the database and egress rules would have denied nearly every entry point across the 2026 incidents this way.

See the incident-by-incident prevention analysis Read the Hugging Face breach case

The honest fine print — the same two assumptions we publish, plus two operational ones

  1. The policy engine must see every request — an agent with raw socket access or a second network path bypasses everything; enforcement belongs at the egress proxy/network layer, not only in an SDK hook.
  2. Default-deny must be on. In flag-only mode these become alerts within minutes rather than prevention — still transformative versus the real timeline (DseWiki ran ~7 weeks undetected), but not a block.
  3. For full URL+method matching on HTTPS you need to be the proxy or in-process hook — SNI alone shows only the host, which still catches the entire host-list layer.
  4. Policy can’t read intent inside a legitimately allowed action: an agent whose job is publishing packages keeps registry access. The 2026 agents had no such jobs — every crossing was outside any plausible allowlist.
Go deeper

Related reference pages

FAQ

API overview questions, answered

What is the page-type lookup API, in one sentence?
One authenticated HTTP GET endpoint that takes a domain or full URL and returns an allow/deny verdict plus the domain's verified page-type map, IAB and filtering categories, and popularity score — evaluated against the database, the egress rules, and the host list on every call.
How much does API access cost?
Plans start at $99/month for the Pro tier (90,000 lookups/month, roughly $1.10 per 1,000 lookups), rising through Pro Plus ($249/mo, 225,000), $499/mo (450,000), $999/mo (900,000), and $1,997/mo (2,000,000, $1.00 per 1,000). Custom volumes are available beyond 10 million lookups a month.
How do I authenticate?
Send your API key as either the X-API-Key HTTP header (recommended for production) or an api_key query parameter. Keys are issued the moment your subscription is activated.
What are the rate limits on the Pro plan?
The Pro plan's monthly quota is 90,000 lookups, resetting on a 30-day cycle from your subscription date. Independent of that quota, a per-IP burst guard of roughly 240 requests per minute applies to every plan, returning 429 if exceeded without consuming quota.
What happens on the Pro plan once 90,000 lookups are used?
Further requests return 403 with an explanatory message until the next 30-day cycle begins, or immediately if you upgrade to a larger plan. There is no automatic overage billing.
Is there a free way to try the API surface before subscribing?
Yes — the free 100-domain sample CSV mirrors the exact record structure an API lookup returns, so you can inspect real fields and verified URLs before choosing a plan.

Get a Pro-plan key and make your first lookup today

$99/month for 90,000 lookups. Subscribe, pay by PayPal or card, and your key is active the moment payment completes.

View Plans & Subscribe