Head-to-head
Cloudflare vs Vercel
In short
Cloudflare publishes a no-training preference into robots.txt with Disallow AI Training (2026-09-15); Vercel's AI Bots ruleset logs or blocks AI crawlers and AI assistants separately (2026-10-29) without writing robots.txt. On charging the rubric grades Cloudflare partial and Vercel not published, and on commerce agents the rubric grades Cloudflare partial and Vercel not published.
The sourced difference
Cloudflare publishes a no-training preference into robots.txt with Disallow AI Training (2026-09-15); Vercel's AI Bots ruleset logs or blocks AI crawlers and AI assistants separately (2026-10-29) without writing robots.txt. Source, read 2026-11-04.
Side by side
| Question | Cloudflare | Vercel |
|---|---|---|
| Tells a verified agent from a spoof | Yes Web Bot Auth signatures, IP validation, verified bot and signed agent directory (BotBase) |
Yes IP, reverse DNS and Web Bot Auth; directory at bots.fyi |
| Per-purpose policy (train, search, answer, act) | Yes separate Search, Training and Agent controls on all plans, plus Disallow AI Training since 2026-09-15 |
Yes AI crawlers and AI assistants handled separately since 2026-10-29 |
| Charging for access | Partial Pay per crawl with HTTP 402, Cloudflare as Merchant of Record, private beta |
Not published support (checked 2026-11-04) |
| Lets signed commerce agents through | Partial Visa TAP and Mastercard Agent Pay built on Web Bot Auth, managed rulesets announced Oct 2025; current rule name not confirmed in docs (checked 2026-11-04) |
Not published Visa TAP or Mastercard Agent Pay support (checked 2026-11-04) |
| Cost | free tier covers AI Crawl Control; Pro from USD 20/month; Bot Management and BotBase are Enterprise and sales-led | published: Basic free; Deep Analysis USD 1 per 1,000 checks on Pro |
| Reporting | Yes AI Crawl Control analytics on crawler activity, request patterns and robots.txt compliance |
Yes Firewall tab BotID filter and Observability Plus metrics |
| Where it runs | edge: enforced on Cloudflare's reverse proxy, no app code needed | both: managed rulesets at Vercel's edge firewall; BotID verdict read in server code via checkBotId() |
| Points, rank | 8 of 10, rank 1 (tied) | 6 of 10, rank 6 (tied) |
| Price read 2026-11-04 | AI Crawl Control and the Search, Training and Agent controls are on all plans including Free (USD 0). Pro USD 20/month billed annually or USD 25 monthly; Business USD 200/month annually or USD 250 monthly; Enterprise Bot Management and BotBase are sales-led. Pay per crawl prices are set by the site owner (private beta). | BotID Basic free on all plans; Deep Analysis USD 1 per 1,000 checkBotId() calls on Pro, custom on Enterprise; AI Bots managed ruleset on all plans. |
Verdicts by reader
- Platform engineer
- Pick by host: both are zone or project toggles.
- Security engineer
- Both verify Web Bot Auth; Vercel exposes isVerifiedBot in checkBotId().
- Publisher operations
- Cloudflare keeps Googlebot, Applebot and Bingbot in search while refusing training; Vercel blocks by category.
What each does not do
Cloudflare
- BotBase does not block traffic itself; mitigation needs Security rules or AI traffic options (BotBase docs, updated 2026-10-08).
- Disallow AI Training cannot pass a no-training preference to Bing until Microsoft ships robots.txt support, targeted early 2027 (Cloudflare blog, 2026-09-15).
- Web Bot Auth verification does no nonce validation today; Cloudflare relies on short expires values (Web Bot Auth docs).
- Pay per crawl is not self-serve: the docs (updated 2026-07-28) tell site owners to reach out to Cloudflare to join the beta.
Vercel
- No charging or pay-per-crawl for AI crawlers (BotID and firewall docs read 2026-11-04).
- BotID Deep Analysis is not on the Hobby plan; it is listed for Pro and Enterprise only (BotID pricing table).
- BotID does not inspect passive page views; it only runs where the app calls checkBotId() (BotID docs).
More: alternatives to Cloudflare, alternatives to Vercel.
Sources
- vercel.com: block ai crawlers and assistants independently with managed rules
- cloudflare.com: network cdn.md
- vercel.com: botid
- developers.cloudflare.com: verified bots
- vercel.com: vercels bot verification now supports web bot auth
- Have it both ways: stay discoverable in search while disallowing AI training
- developers.cloudflare.com: what is pay per crawl
- Securing agentic commerce: helping AI Agents transact with Visa and Mastercard
- vercel.com: verified bots
Last reviewed 2026-11-04