Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Browserbase: 2 rows

In short

Browserbase has 2 rows in the Index: Browserbase cloud browsers and Stagehand. They cover 1 purpose: task agent. 0 of them have a robots.txt token and 1 publish nothing a site can verify. The crawl measured none of its tokens. Key directory; signing is a customer feature.

Browserbase: 2 rows.

Operator Token Purpose Respects robots.txt Verification Block rate 2026-11-04 Profile
Browserbase Browserbase cloud browsers no token Task agent No docs Web Bot Auth no token Profile
Browserbase Stagehand (open-source framework) no token Task agent No docs None published no token Profile

Vendor profile: Browserbase, with its products, standards, our probes and what it does not do.

Verification Ledger entry

Summary
Key directory; signing is a customer feature.
IP lists
publishes nothing
Reverse DNS
publishes nothing
Key directory
https://www.browserbase.com/.well-known/http-message-signatures-directory
Cloudflare
Cloudflare signed agent (cohort of 2025-08-28)
Visa and Mastercard
Visa TAP: no public operator-level registration found (checked 2026-11-04). Mastercard Agent Pay: no public operator-level registration found (checked 2026-11-04).

Sources

Last reviewed 2026-11-04