Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

News from 2026-08-28

Cloudflare opens BotBase for Operators

In short

28 August 2026: Cloudflare opens BotBase for Operators. Bot operators get a dashboard home for submissions under Application Security, with status (waiting, accepted, rejected with reasons), editing of existing entries, and a behaviour model to declare how a bot uses content. Cloudflare says accurate entries decide whether a bot earns and keeps Verified status. Rows affected in this site: Cloudflare.

Bot operators get a dashboard home for submissions under Application Security, with status (waiting, accepted, rejected with reasons), editing of existing entries, and a behaviour model to declare how a bot uses content. Cloudflare says accurate entries decide whether a bot earns and keeps Verified status.

Primary source blog.cloudflare.com/botbase-for-operators/, read 2026-11-04.

Where it stands in the Index

Cloudflare
Vendor. CDN and security network that sorts crawlers and agents by purpose (search, training, agent), verifies them by IP or Web Bot Auth signatures, and offers per-request charging for AI crawlers in private beta.

All news.

Sources

Last reviewed 2026-11-04