---
title: "Anthropic crawlers and agents | Agent Ready Crawlers"
description: "Anthropic in the Crawler Index: 7 crawlers and agents with tokens, purposes, verification method and block rates, read 2026-11-04."
canonical: https://agentreadycrawlers.com/crawlers/operator/anthropic
site: "Agent Ready Crawlers"
last_updated: 2026-11-04
section: "Crawlers"
---

# Anthropic crawlers and agents | Agent Ready Crawlers

> Anthropic in the Crawler Index: 7 crawlers and agents with tokens, purposes, verification method and block rates, read 2026-11-04.

Source: https://agentreadycrawlers.com/crawlers/operator/anthropic
Site: Agent Ready Crawlers
Last reviewed: 2026-11-04

# Anthropic: 7 rows

In short

Anthropic has 7 rows in the Index: ClaudeBot, Claude-User, Claude-SearchBot, anthropic-ai, Claude in Chrome, Claude Duet cloud browser and Claude computer use. They cover 4 purposes: training, live answer fetch, search index, task agent. 5 of them have a robots.txt token and 4 publish nothing a site can verify.

Anthropic: 7 rows.

| Operator | Token | Purpose | Respects robots.txt | Verification | Block rate 2026-11-04 | Profile |
| --- | --- | --- | --- | --- | --- | --- |
| Anthropic | ClaudeBot | Training | Yes | IP list | 17.6% | Profile |
| Anthropic | Claude-User | Live answer fetch | Yes | IP list | 8.1% | Profile |
| Anthropic | Claude-SearchBot | Search index | Yes | IP list | 7.7% | Profile |
| Anthropic | anthropic-ai | Training | Legacy | None published | 15.5% | Profile |
| Anthropic | Claude in Chrome no token | Task agent | No statement | None published | no token | Profile |
| Anthropic | Claude-Agent | Task agent | Reported | None published | not measured | Profile |
| Anthropic | Claude computer use (API tool) no token | Task agent | No docs | None published | no token | Profile |

Vendor profile: [Anthropic](https://agentreadycrawlers.com/vendors/anthropic/), with its products, standards, our probes and what it does not do.

## Verification Ledger entry

Summary

One shared IP list for all three bots; no key directory at claude.ai or claude.com (404, 2026-11-04); nothing for Claude in Chrome.

IP lists

-   [claude.com/crawling/bots.json](https://claude.com/crawling/bots.json)

Reverse DNS

publishes nothing

Key directory

publishes nothing

Cloudflare

not confirmed on Radar (directory slug not found, 2026-11-04)

Visa and Mastercard

Visa TAP: no public operator-level registration found (checked 2026-11-04). Mastercard Agent Pay: no public operator-level registration found (checked 2026-11-04).

## Block rates for Anthropic tokens

| Token | Refused, top 10,000 | Named | Via * | All crawled |
| --- | --- | --- | --- | --- |
| ClaudeBot | 17.6% | 14.0% | 3.5% | 13.1% |
| anthropic-ai | 15.5% | 11.7% | 3.8% | 11.6% |
| Claude-User | 8.1% | 4.5% | 3.6% | 5.7% |
| Claude-SearchBot | 7.7% | 4.1% | 3.6% | 5.5% |

Based on 5,938 top-10,000 sites with a parsed robots.txt; crawl of 2026-11-04.

## In the news

-   2026-10-21[Anthropic's Claude Duet starts browsing with a Claude-Agent token](https://agentreadycrawlers.com/news/2026-10-21-anthropic-s-claude-duet-starts-browsing-with-a-claude-agent/)
-   2026-10-20[Anthropic refreshes its crawler IP list](https://agentreadycrawlers.com/news/2026-10-20-anthropic-refreshes-its-crawler-ip-list/)

## Sources

-   [Does Anthropic crawl data from the web, and how can site owners block the crawler?](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler) read 2026-11-04
-   [claude.com: bots (JSON)](https://claude.com/crawling/bots.json) read 2026-11-04
-   [claude.com: browser](https://claude.com/docs/third-party/claude-desktop/browser) read 2026-11-04
-   [support.claude.com: 12012173 getting started with claude in chrome](https://support.claude.com/en/articles/12012173-getting-started-with-claude-in-chrome) read 2026-11-04

Last reviewed 2026-11-04