Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Browsing agent from Cloudflare

Cloudflare Browser Rendering

In short

Cloudflare Browser Rendering is Cloudflare's browsing agent. Cloudflare Browser Rendering has no robots.txt token, so robots.txt cannot address it and the crawl cannot measure a block rate. It signs requests with Web Bot Auth, so a signature can be checked. Cloudflare documents the automatic request headers; robots.txt handling is up to the customer's script.

Operator
Cloudflare Vendor profile: Cloudflare
Purpose
Task agent All task agent rows. A browsing agent that clicks, fills forms and finishes a job for a person.
User-agent string
No exact string is published. What the operator or a third party says: Sends signed Web Bot Auth headers
robots.txt token
None. No robots.txt line can address this agent.
Respects robots.txt

No statement

Cloudflare documents the automatic request headers; robots.txt handling is up to the customer's script. Source, read 2026-11-04.

Verification

Web Bot Auth signature

Requests carry an HTTP message signature that can be checked against the operator's published key directory. Documentation, read 2026-11-04.

IP ranges
Not published.
Rate behaviour
Set by the customer.
Block rate

None: the agent has no token, so a robots.txt block rate does not exist.

Documentation
blog.cloudflare.com, read 2026-11-04.

Note. Classed as a signed agent since 2025-08-28; always given bot score 1 for Bot Management customers, per Cloudflare.

Allow and refuse lines

This agent has no robots.txt token, so there are no lines to copy. Verify its signature in your WAF or edge rules instead, then allow or refuse the verified identity.

Refusing or allowing in robots.txt states a preference. For this row the operator or its documentation says robots.txt may not govern the fetch, so enforcement belongs in your WAF.

Cloudflare in the Verification Ledger

Signs Browser Rendering traffic. All operators.

IP lists
publishes nothing
Reverse DNS
publishes nothing
Key directory
documented in Browser Rendering automatic request headers
Cloudflare
Cloudflare signed agent
Visa TAP registry
no public operator-level registration found (checked 2026-11-04)
Mastercard Agent Pay
no public operator-level registration found (checked 2026-11-04)

Sources

Last reviewed 2026-11-04