---
title: "FirecrawlAgent: user agent, token, block rate | Agent Ready Crawlers"
description: "FirecrawlAgent from Firecrawl: robots.txt token FirecrawlAgent, other purpose and what a site can check; no exact user-agent string is published."
canonical: https://agentreadycrawlers.com/crawlers/firecrawl
site: "Agent Ready Crawlers"
last_updated: 2026-11-04
section: "Crawlers"
---

# FirecrawlAgent: user agent, token, block rate | Agent Ready Crawlers

> FirecrawlAgent from Firecrawl: robots.txt token FirecrawlAgent, other purpose and what a site can check; no exact user-agent string is published.

Source: https://agentreadycrawlers.com/crawlers/firecrawl
Site: Agent Ready Crawlers
Last reviewed: 2026-11-04

Bot from Firecrawl

# FirecrawlAgent

In short

FirecrawlAgent is Firecrawl's bot, filed under Other. The token to write in robots.txt is FirecrawlAgent. The operator publishes nothing a site can check. The crawl of 2026-11-04 did not measure FirecrawlAgent. Firecrawl advises allowlisting with robots.txt rather than a WAF rule on User-Agent. FirecrawlAgent reads robots.txt before it crawls and applies rules for FirecrawlAgent and for \*.

Operator

Firecrawl Vendor profile: [Firecrawl](https://agentreadycrawlers.com/vendors/firecrawl/)

Purpose

Other [All other rows](https://agentreadycrawlers.com/crawlers/purpose/other/). Link previews, ad checks, dataset crawls, SEO tools and legacy tokens that fit none of the five purposes.

User-agent string

No exact string is published. What the operator or a third party says: Not fixed: Firecrawl says do not assume FirecrawlAgent is the User-Agent header on the fetch

robots.txt token

`FirecrawlAgent` Matched case-insensitively (RFC 9309). The token and the user-agent string are different things.

Respects robots.txt

Yes, per operator

Firecrawl: reads robots.txt before it crawls and applies rules for FirecrawlAgent and for \*. [Source](https://docs.firecrawl.dev/advanced-scraping-guide), read 2026-11-04.

Verification

Nothing published

The operator publishes no list, reverse DNS pattern or key directory a site owner can check. [Documentation](https://docs.firecrawl.dev/advanced-scraping-guide), read 2026-11-04.

IP ranges

Not published.

Rate behaviour

Set by the customer job.

Block rate

Not measured: the crawl checked 31 AI and search tokens, and `FirecrawlAgent` is not one of them.

Documentation

[docs.firecrawl.dev](https://docs.firecrawl.dev/advanced-scraping-guide), read 2026-11-04.

Note. Firecrawl advises allowlisting with robots.txt rather than a WAF rule on User-Agent. GitHub issue #1169 records complaints that the token never appears in logs.

## Allow and refuse lines

### To allow FirecrawlAgent

User-agent: FirecrawlAgent
Allow: /

### To refuse FirecrawlAgent

User-agent: FirecrawlAgent
Disallow: /

Refusing or allowing in robots.txt states a preference. Enforcement of a refusal belongs in your WAF; see how to verify the identity above.

## Firecrawl in the Verification Ledger

Robots.txt token only; Firecrawl advises against UA-based allowlisting. [All operators](https://agentreadycrawlers.com/verification/).

IP lists

publishes nothing

Reverse DNS

publishes nothing

Key directory

publishes nothing

Cloudflare

not checked

Visa TAP registry

no public operator-level registration found (checked 2026-11-04)

Mastercard Agent Pay

no public operator-level registration found (checked 2026-11-04)

## Sources

-   [docs.firecrawl.dev: advanced scraping guide](https://docs.firecrawl.dev/advanced-scraping-guide) read 2026-11-04

Last reviewed 2026-11-04