News from 2026-10-20

Anthropic refreshes its crawler IP list

In short

20 October 2026: Anthropic refreshes its crawler IP list. claude.com/crawling/bots.json, the list Anthropic links for verifying ClaudeBot, Claude-User and Claude-SearchBot, carried creationTime 2026-10-20 when read, starting with 216.73.216.0/22. Anthropic warns that IP blocking may stop its bots reading robots.txt. Claude-User is Anthropic's live answer fetcher. Rows affected in this site: ClaudeBot, Claude-User and Claude-SearchBot.

claude.com/crawling/bots.json, the list Anthropic links for verifying ClaudeBot, Claude-User and Claude-SearchBot, carried creationTime 2026-10-20 when read, starting with 216.73.216.0/22. Anthropic warns that IP blocking may stop its bots reading robots.txt.

Primary source claude.com/crawling/bots.json, read 2026-11-04.

Also read

• Anthropic support: does Anthropic crawl data from the web, and how can site owners block the crawler. Anthropic says blocking the IP addresses its bots operate from may not work correctly or persistently guarantee an opt-out, because it impedes its ability to read your robots.txt. Read 2026-11-04.

Where it stands in the Index

ClaudeBot

Anthropic, training. Token ClaudeBot. Verification: ip list. Refused by 17.6% of top-10,000 sites.

Claude-User

Anthropic, live answer fetch. Token Claude-User. Verification: ip list. Refused by 8.1% of top-10,000 sites.

Claude-SearchBot

Anthropic, search index. Token Claude-SearchBot. Verification: ip list. Refused by 7.7% of top-10,000 sites.

All news.

Last reviewed 2026-11-04