---
title: "Meta crawlers and agents | Agent Ready Crawlers"
description: "Meta in the Crawler Index: 7 crawlers and agents with tokens, purposes, verification method and block rates, read 2026-11-04."
canonical: https://agentreadycrawlers.com/crawlers/operator/meta
site: "Agent Ready Crawlers"
last_updated: 2026-11-04
section: "Crawlers"
---

# Meta crawlers and agents | Agent Ready Crawlers

> Meta in the Crawler Index: 7 crawlers and agents with tokens, purposes, verification method and block rates, read 2026-11-04.

Source: https://agentreadycrawlers.com/crawlers/operator/meta
Site: Agent Ready Crawlers
Last reviewed: 2026-11-04

# Meta: 7 rows

In short

Meta has 7 rows in the Index: Meta-ExternalAgent, Meta-ExternalFetcher, Meta-WebIndexer, Meta-ExternalAds, FacebookExternalHit, FacebookBot and Muse. They cover 5 purposes: training, live answer fetch, search index, other, shopping agent. 6 of them have a robots.txt token and 7 publish nothing a site can verify. The most refused is Meta-ExternalAgent, disallowed at the root by 16.0 percent of top-10,000 sites.

Meta: 7 rows.

| Operator | Token | Purpose | Respects robots.txt | Verification | Block rate 2026-11-04 | Profile |
| --- | --- | --- | --- | --- | --- | --- |
| Meta | meta-externalagent | Training | Yes | None published | 16.0% | Profile |
| Meta | meta-externalfetcher | Live answer fetch | May not apply | None published | 8.5% | Profile |
| Meta | meta-webindexer | Search index | Yes | None published | not measured | Profile |
| Meta | meta-externalads | Other | Yes | None published | not measured | Profile |
| Meta | facebookexternalhit | Other | May not apply | None published | not measured | Profile |
| Meta | FacebookBot | Other | Legacy | None published | 10.2% | Profile |
| Meta | Muse (personal agent) no token | Shopping agent | No docs | None published | no token | Profile |

Vendor profile: [Meta](https://agentreadycrawlers.com/vendors/meta/), with its products, standards, our probes and what it does not do.

## Verification Ledger entry

Summary

User agent strings only; the page recommends IP allowlisting but links no list. Publishes nothing for Muse.

IP lists

publishes nothing

Reverse DNS

publishes nothing

Key directory

publishes nothing

Cloudflare

not checked

Visa and Mastercard

Visa TAP: no public operator-level registration found (checked 2026-11-04). Mastercard Agent Pay: no public operator-level registration found (checked 2026-11-04).

## Block rates for Meta tokens

| Token | Refused, top 10,000 | Named | Via * | All crawled |
| --- | --- | --- | --- | --- |
| Meta-ExternalAgent | 16.0% | 12.3% | 3.7% | 12.0% |
| FacebookBot | 10.2% | 6.4% | 3.7% | 7.3% |
| Meta-ExternalFetcher | 8.5% | 4.7% | 3.8% | 6.0% |

Based on 5,938 top-10,000 sites with a parsed robots.txt; crawl of 2026-11-04.

## In the news

-   2026-10-15[DataDome Q3 report: 25 billion AI agent requests](https://agentreadycrawlers.com/news/2026-10-15-datadome-q3-report-25-billion-ai-agent-requests/)
-   2026-09-08[Meta launches Muse, a personal agent that does not identify itself](https://agentreadycrawlers.com/news/2026-09-08-meta-launches-muse-a-personal-agent-that-does-not-identify-i/)

## Sources

-   [Meta Web Crawlers](https://developers.facebook.com/docs/sharing/webmasters/web-crawlers/) read 2026-11-04

Last reviewed 2026-11-04