Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Microsoft: 3 rows

In short

Microsoft has 3 rows in the Index: Bingbot, MicrosoftPreview and Browse with Copilot. They cover 3 purposes: search index, other, task agent. 2 of them have a robots.txt token and 1 publish nothing a site can verify. The most refused is MicrosoftPreview, disallowed at the root by 4.0 percent of top-10,000 sites.

Microsoft: 3 rows.

Operator Token Purpose Respects robots.txt Verification Block rate 2026-11-04 Profile
Microsoft bingbot Search index Yes Reverse DNS 3.1% Profile
Microsoft MicrosoftPreview Other Yes Reverse DNS 4.0% Profile
Microsoft Browse with Copilot (formerly Copilot Actions) no token Task agent No docs None published no token Profile

Vendor profile: Microsoft, with its products, standards, our probes and what it does not do.

Verification Ledger entry

Summary
bingbot.json (creationTime 2024-01-03) and a verification tool; nothing for Browse with Copilot.
IP lists
Reverse DNS
Verify Bingbot tool and documented methods
Key directory
not published
Cloudflare
not checked
Visa and Mastercard
Visa TAP: no public operator-level registration found (checked 2026-11-04). Mastercard Agent Pay: no public operator-level registration found (checked 2026-11-04).

Block rates for Microsoft tokens

TokenRefused, top 10,000NamedVia *All crawled
MicrosoftPreview4.0%0.0%3.9%2.9%
Bingbot3.1%0.4%2.7%2.2%

Based on 5,938 top-10,000 sites with a parsed robots.txt; crawl of 2026-11-04.

In the news

Sources

Last reviewed 2026-11-04