Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Bot from Meta

FacebookBot (legacy token)

In short

FacebookBot (legacy token) is Meta's bot, filed under Other. The token to write in robots.txt is FacebookBot. The operator publishes nothing a site can check. Still named in many robots.txt files (twitter.com names it in the shared crawl, 2026-11-04). On robots.txt, not listed on Meta's current crawler page, which names FacebookExternalHit, Meta-WebIndexer, Meta-ExternalAds, Meta-ExternalAgent and Meta-ExternalFetcher (checked 2026-11-04).

Operator
Meta Vendor profile: Meta
Purpose
Other All other rows. Link previews, ad checks, dataset crawls, SEO tools and legacy tokens that fit none of the five purposes.
User-agent string
No exact string is published. What the operator or a third party says: Not listed on Meta's current crawler page
robots.txt token
FacebookBot Matched case-insensitively (RFC 9309). The token and the user-agent string are different things.
Respects robots.txt

Not on current operator page

Not listed on Meta's current crawler page, which names FacebookExternalHit, Meta-WebIndexer, Meta-ExternalAds, Meta-ExternalAgent and Meta-ExternalFetcher (checked 2026-11-04). Source, read 2026-11-04.

Verification

Nothing published

The operator publishes no list, reverse DNS pattern or key directory a site owner can check. Documentation, read 2026-11-04.

IP ranges
Not published.
Rate behaviour
Not documented.
Block rate

10.2% of 5,938 top-10,000 sites with a parsed robots.txt disallow FacebookBot at the root (603 sites). 6.4% name it; 3.7% refuse it only through a * group. Rank 18 of 31 tokens measured.

Across all 8,624 parsed robots.txt files in the crawl: 7.3% (4.6% named). Crawl of 2026-11-04; report.

Documentation
developers.facebook.com, read 2026-11-04.

Note. Still named in many robots.txt files (twitter.com names it in the shared crawl, 2026-11-04). Kept as a legacy row.

Allow and refuse lines

To allow FacebookBot

User-agent: FacebookBot
Allow: /

To refuse FacebookBot

User-agent: FacebookBot
Disallow: /

Refusing or allowing in robots.txt states a preference. Enforcement of a refusal belongs in your WAF; see how to verify the identity above.

Meta in the Verification Ledger

User agent strings only; the page recommends IP allowlisting but links no list. Publishes nothing for Muse. All operators.

IP lists
publishes nothing
Reverse DNS
publishes nothing
Key directory
publishes nothing
Cloudflare
not checked
Visa TAP registry
no public operator-level registration found (checked 2026-11-04)
Mastercard Agent Pay
no public operator-level registration found (checked 2026-11-04)

Other Meta rows

Sources

Last reviewed 2026-11-04