Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Glossary

Pay per crawl

In short

Cloudflare's closed-beta feature that lets a zone charge AI crawlers per request using HTTP 402, with Cloudflare as Merchant of Record. The crawl saw 28 home pages answer 402, 0.3 percent of those fetched. A 402 is not proof of pay per crawl, which is in closed beta at Cloudflare. Related: What pay per crawl pays and TollBit vs Cloudflare.

Definition

Cloudflare's closed-beta feature that lets a zone charge AI crawlers per request using HTTP 402, with Cloudflare as Merchant of Record.

Sourced example

A crawler gets 402 with crawler-price: USD XX.XX and can retry with crawler-exact-price. Source, read 2026-11-04.

How it shows up here

The crawl saw 28 home pages answer 402, 0.3 percent of those fetched. A 402 is not proof of pay per crawl, which is in closed beta at Cloudflare.

Previous: nosnippet. Next: Reverse DNS verification. All 41 terms.

Sources

Last reviewed 2026-11-04