Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Browsing agent from Browserbase

Stagehand (open-source framework)

In short

Stagehand (open-source framework) is Browserbase's browsing agent. Stagehand has no robots.txt token, so robots.txt cannot address it and the crawl cannot measure a block rate. The operator publishes nothing a site can check. Stagehand has no operator bot documentation, checked 2026-11-04. Identity comes from the hosting browser, not the framework.

Operator
Browserbase Vendor profile: Browserbase
Purpose
Task agent All task agent rows. A browsing agent that clicks, fills forms and finishes a job for a person.
User-agent string
No exact string is published. What the operator or a third party says: Inherits the browser it runs in; signed only when run on Browserbase with Web Bot Auth
robots.txt token
None. No robots.txt line can address this agent.
Respects robots.txt

No documentation

operator publishes no bot documentation (checked 2026-11-04) for Stagehand itself. Source, read 2026-11-04.

Operator publishes no bot documentation (checked 2026-11-04).

Verification

Nothing published

The operator publishes no list, reverse DNS pattern or key directory a site owner can check. Documentation, read 2026-11-04.

IP ranges
Not published.
Rate behaviour
Set by the developer.
Block rate

None: the agent has no token, so a robots.txt block rate does not exist.

Documentation
Operator publishes no bot documentation (checked 2026-11-04). Closest page read.

Note. Identity comes from the hosting browser, not the framework.

Allow and refuse lines

This agent has no robots.txt token, so there are no lines to copy. Without a token, a published list or a signature, a site owner cannot tell it from an ordinary visitor by request alone.

Refusing or allowing in robots.txt states a preference. For this row the operator or its documentation says robots.txt may not govern the fetch, so enforcement belongs in your WAF.

Browserbase in the Verification Ledger

Key directory; signing is a customer feature. All operators.

IP lists
publishes nothing
Reverse DNS
publishes nothing
Key directory
https://www.browserbase.com/.well-known/http-message-signatures-directory
Cloudflare
Cloudflare signed agent (cohort of 2025-08-28)
Visa TAP registry
no public operator-level registration found (checked 2026-11-04)
Mastercard Agent Pay
no public operator-level registration found (checked 2026-11-04)

Other Browserbase rows

Sources

Last reviewed 2026-11-04