Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Head-to-head

Arcjet vs Vercel

In short

Arcjet verifies known bots by IP and reverse DNS inside application code through its SDK; Vercel verifies by IP, reverse DNS and Web Bot Auth at its edge and splits AI crawlers from AI assistants in a managed ruleset since 2026-10-29. For a security engineer: Vercel accepts Web Bot Auth signatures.

The sourced difference

Arcjet verifies known bots by IP and reverse DNS inside application code through its SDK; Vercel verifies by IP, reverse DNS and Web Bot Auth at its edge and splits AI crawlers from AI assistants in a managed ruleset since 2026-10-29. Source, read 2026-11-04.

Side by side

QuestionArcjetVercel
Tells a verified agent from a spoof Partial
IP data and reverse DNS for known bots; Web Bot Auth not mentioned in docs (checked 2026-11-04)
Yes
IP, reverse DNS and Web Bot Auth; directory at bots.fyi
Per-purpose policy (train, search, answer, act) Partial
allow or deny by bot name or category (e.g. CATEGORY:AI) per route
Yes
AI crawlers and AI assistants handled separately since 2026-10-29
Charging for access Not published
support (checked 2026-11-04)
Not published
support (checked 2026-11-04)
Lets signed commerce agents through Not published
Visa TAP or Mastercard Agent Pay support (checked 2026-11-04)
Not published
Visa TAP or Mastercard Agent Pay support (checked 2026-11-04)
Cost published: from USD 25/month plus USD 5 per 1M web requests published: Basic free; Deep Analysis USD 1 per 1,000 checks on Pro
Reporting Partial
request decision logs with 1 to 24 hours retention on self-serve plans
Yes
Firewall tab BotID filter and Observability Plus metrics
Where it runs app: SDK calls in application code both: managed rulesets at Vercel's edge firewall; BotID verdict read in server code via checkBotId()
Points, rank3 of 10, rank 11 (tied)6 of 10, rank 6 (tied)
Price read 2026-11-04Individual USD 25/month, Startup USD 299/month, Enterprise custom; usage USD 5 per 1M web requests and USD 50 per 1M agent requests; free plan 10,000 requests/month (hard stop) after a 15-day trial.BotID Basic free on all plans; Deep Analysis USD 1 per 1,000 checkBotId() calls on Pro, custom on Enterprise; AI Bots managed ruleset on all plans.

Verdicts by reader

Platform engineer
On Vercel hosting, the managed ruleset is a toggle; Arcjet works on any host but every rule is code.
Security engineer
Vercel accepts Web Bot Auth signatures; Arcjet docs do not mention Web Bot Auth (checked 2026-11-04).
Publisher operations
Vercel can block training crawlers while keeping answer fetchers; Arcjet can do it per route with CATEGORY rules.

What each does not do

Arcjet

  • No Web Bot Auth or cryptographic agent verification in bot protection docs (checked 2026-11-04).
  • Does not run at the CDN edge; requests reach the app before Arcjet decides (concepts page).
  • No AI crawler monetization or pay-per-crawl (docs read 2026-11-04).

Vercel

  • No charging or pay-per-crawl for AI crawlers (BotID and firewall docs read 2026-11-04).
  • BotID Deep Analysis is not on the Hobby plan; it is listed for Pro and Enterprise only (BotID pricing table).
  • BotID does not inspect passive page views; it only runs where the app calls checkBotId() (BotID docs).

More: alternatives to Arcjet, alternatives to Vercel.

Sources

Last reviewed 2026-11-04