Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Browsing agent from Perplexity

Comet (browser assistant)

In short

Comet (browser assistant) is Perplexity's browsing agent. Comet has no robots.txt token, so robots.txt cannot address it and the crawl cannot measure a block rate. The operator publishes nothing a site can check. Comet has no operator bot documentation, checked 2026-11-04. Runs on the user's device and IP.

Operator
Perplexity Vendor profile: Perplexity
Purpose
Task agent All task agent rows. A browsing agent that clicks, fills forms and finishes a job for a person.
User-agent string
No exact string is published. What the operator or a third party says: Chrome's user agent from the user's own device; no unique identifier
robots.txt token
None. No robots.txt line can address this agent.
Respects robots.txt

No documentation

operator publishes no bot documentation (checked 2026-11-04) for Comet's agent; it browses as the user's Chromium browser. Source, read 2026-11-04.

Operator publishes no bot documentation (checked 2026-11-04).

Verification

Nothing published

The operator publishes no list, reverse DNS pattern or key directory a site owner can check. Documentation, read 2026-11-04.

IP ranges
Not published.
Rate behaviour
Runs on the user's device and IP.
Block rate

None: the agent has no token, so a robots.txt block rate does not exist.

Documentation
Operator publishes no bot documentation (checked 2026-11-04). Closest page read.

Note. Amazon's 2025-11-04 suit argued Comet sends Chrome's user agent with no unique identifier; the Ninth Circuit vacated the preliminary injunction on 2026-08-04, per Verisoul's summary of the record.

Allow and refuse lines

This agent has no robots.txt token, so there are no lines to copy. Without a token, a published list or a signature, a site owner cannot tell it from an ordinary visitor by request alone.

Refusing or allowing in robots.txt states a preference. For this row the operator or its documentation says robots.txt may not govern the fetch, so enforcement belongs in your WAF.

Perplexity in the Verification Ledger

Two IP lists and WAF recipes; no key directory (404 at www.perplexity.ai, 2026-11-04); nothing for Comet. All operators.

IP lists
Reverse DNS
publishes nothing
Key directory
publishes nothing
Cloudflare
not checked
Visa TAP registry
no public operator-level registration found (checked 2026-11-04)
Mastercard Agent Pay
no public operator-level registration found (checked 2026-11-04)

Other Perplexity rows

Sources

Last reviewed 2026-11-04