Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Bot from Meta

FacebookExternalHit

In short

FacebookExternalHit is Meta's bot, filed under Other. The token to write in robots.txt is facebookexternalhit. The operator publishes nothing a site can check. The crawl of 2026-11-04 did not measure facebookexternalhit. Meta suggests allowlisting by IP as the more secure option but the page links no IP list. FacebookExternalHit might bypass robots.txt when performing security or integrity checks.

Operator
Meta Vendor profile: Meta
Purpose
Other All other rows. Link previews, ad checks, dataset crawls, SEO tools and legacy tokens that fit none of the five purposes.
User-agent string
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
robots.txt token
facebookexternalhit Matched case-insensitively (RFC 9309). The token and the user-agent string are different things.
Respects robots.txt

May not apply (user-triggered)

Meta: might bypass robots.txt when performing security or integrity checks. Source, read 2026-11-04.

Verification

Nothing published

The operator publishes no list, reverse DNS pattern or key directory a site owner can check. Documentation, read 2026-11-04.

IP ranges
Not published.
Rate behaviour
Requests may use a Range header; Open Graph tags must sit in the first 1 MB.
Block rate

Not measured: the crawl checked 31 AI and search tokens, and facebookexternalhit is not one of them.

Documentation
developers.facebook.com, read 2026-11-04.

Note. Link-preview fetcher for shared links. Meta suggests allowlisting by IP as the more secure option but the page links no IP list.

Allow and refuse lines

To allow facebookexternalhit

User-agent: facebookexternalhit
Allow: /

To refuse facebookexternalhit

User-agent: facebookexternalhit
Disallow: /

Refusing or allowing in robots.txt states a preference. For this row the operator or its documentation says robots.txt may not govern the fetch, so enforcement belongs in your WAF.

Meta in the Verification Ledger

User agent strings only; the page recommends IP allowlisting but links no list. Publishes nothing for Muse. All operators.

IP lists
publishes nothing
Reverse DNS
publishes nothing
Key directory
publishes nothing
Cloudflare
not checked
Visa TAP registry
no public operator-level registration found (checked 2026-11-04)
Mastercard Agent Pay
no public operator-level registration found (checked 2026-11-04)

Other Meta rows

Sources

Last reviewed 2026-11-04