Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Live answer fetcher from OpenAI

ChatGPT-User

In short

ChatGPT-User is OpenAI's live answer fetcher. The token to write in robots.txt is ChatGPT-User. Check a visitor against the operator's published IP list. 9.7 percent of 5,938 top-10,000 sites with a parsed robots.txt disallow it at the root (crawl of 2026-11-04). ChatGPT-User: because these actions are initiated by a user, robots.txt rules may not apply.

Operator
OpenAI Vendor profile: OpenAI
Purpose
Live answer fetch All live answer fetch rows. Fetches a page because a person asked an assistant about it, one request at a time.
User-agent string
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ChatGPT-User/1.0; +https://openai.com/bot
robots.txt token
ChatGPT-User Matched case-insensitively (RFC 9309). The token and the user-agent string are different things.
Respects robots.txt

May not apply (user-triggered)

OpenAI: because these actions are initiated by a user, robots.txt rules may not apply. ChatGPT-User is not used to decide search inclusion. Source, read 2026-11-04.

Verification

Published IP list (JSON)

Match the source address of a request against the operator's published list. Documentation, read 2026-11-04.

IP ranges
openai.com/chatgpt-user.json
Rate behaviour
Not used for automatic crawling, per OpenAI. chatgpt-user.json creationTime read: 2026-10-27.
Block rate

9.7% of 5,938 top-10,000 sites with a parsed robots.txt disallow ChatGPT-User at the root (574 sites). 6.3% name it; 3.3% refuse it only through a * group. Rank 20 of 31 tokens measured.

Across all 8,624 parsed robots.txt files in the crawl: 6.9% (4.5% named). Crawl of 2026-11-04; report.

Documentation
platform.openai.com, read 2026-11-04.

Note. Also used for GPT Actions and Custom GPT fetches. Use OAI-SearchBot, not ChatGPT-User, to manage search opt-outs.

Allow and refuse lines

To allow ChatGPT-User

User-agent: ChatGPT-User
Allow: /

To refuse ChatGPT-User

User-agent: ChatGPT-User
Disallow: /

Refusing or allowing in robots.txt states a preference. For this row the operator or its documentation says robots.txt may not govern the fetch, so enforcement belongs in your WAF.

OpenAI in the Verification Ledger

One IP JSON per crawler plus a Web Bot Auth key directory for the Cloud browser. The most complete set read. All operators.

IP lists
Reverse DNS
publishes nothing
Key directory
https://chatgpt.com/.well-known/http-message-signatures-directory
Cloudflare
GPTBot and ChatGPT agent listed Verified on Radar (read 2026-11-04); ChatGPT agent is a Cloudflare signed agent, detection ID 129220581
Visa TAP registry
no public operator-level registration found (checked 2026-11-04)
Mastercard Agent Pay
no public operator-level registration found (checked 2026-11-04)

In the news

Other OpenAI rows

Sources

Last reviewed 2026-11-04