Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

DuckDuckGo: 2 rows

In short

DuckDuckGo has 2 rows in the Index: DuckAssistBot and DuckDuckBot. They cover 2 purposes: live answer fetch, search index. 2 of them have a robots.txt token and 0 publish nothing a site can verify. The most refused is DuckAssistBot, disallowed at the root by 8.1 percent of top-10,000 sites. Two IP lists.

DuckDuckGo: 2 rows.

Operator Token Purpose Respects robots.txt Verification Block rate 2026-11-04 Profile
DuckDuckGo DuckAssistBot Live answer fetch Yes IP list 8.1% Profile
DuckDuckGo DuckDuckBot Search index Yes IP list not measured Profile

Vendor profile: DuckDuckGo, with its products, standards, our probes and what it does not do.

Verification Ledger entry

Summary
Two IP lists.
IP lists
Reverse DNS
publishes nothing
Key directory
publishes nothing
Cloudflare
not checked
Visa and Mastercard
Visa TAP: no public operator-level registration found (checked 2026-11-04). Mastercard Agent Pay: no public operator-level registration found (checked 2026-11-04).

Block rates for DuckDuckGo tokens

TokenRefused, top 10,000NamedVia *All crawled
DuckAssistBot8.1%4.3%3.8%5.8%

Based on 5,938 top-10,000 sites with a parsed robots.txt; crawl of 2026-11-04.

Sources

Last reviewed 2026-11-04