--- title: "Perplexity: crawler operator profile | Agent Ready Crawlers" description: "Perplexity: crawler operator profile covering PerplexityBot and Perplexity-User. Sourced facts, standards, dated prices and limits, read 2026-11-05." canonical: https://agentreadycrawlers.com/vendors/perplexity site: "Agent Ready Crawlers" last_updated: 2026-11-05 section: "Vendors" --- # Perplexity: crawler operator profile | Agent Ready Crawlers > Perplexity: crawler operator profile covering PerplexityBot and Perplexity-User. Sourced facts, standards, dated prices and limits, read 2026-11-05. Source: https://agentreadycrawlers.com/vendors/perplexity Site: Agent Ready Crawlers Last reviewed: 2026-11-05 Crawler operator # Perplexity In short Perplexity splits its traffic between an index crawler and a user-triggered fetcher, ships allow-rule recipes for two WAF products, and publishes JSON range files that sites are told to treat as the source of truth. PerplexityBot is said not to feed foundation model training, while the fetcher generally disregards robots.txt. Type [Crawler operator](https://agentreadycrawlers.com/vendors/by-type/crawler-operator/) Homepage [perplexity.ai](https://www.perplexity.ai) In one line Perplexity documents two identities for site owners, PerplexityBot for its search index and Perplexity-User for fetches a user asks for, each with its own IP file, and publishes allow-rule recipes for Cloudflare WAF and AWS WAF. Last read 2026-11-05 ## What Perplexity does for this layer ### From the vendor's own pages Perplexity says PerplexityBot surfaces and links websites in Perplexity search results and is not used to crawl content for AI foundation models. ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) Perplexity says Perplexity-User visits pages when users ask Perplexity a question and that, since a user requested the fetch, it "generally ignores robots.txt rules". ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) Perplexity's Cloudflare recipe is a custom rule that matches each user agent against its own IP ranges and uses the Skip action, placed above any rule that blocks or challenges. ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) Perplexity notes that Cloudflare's Bot Fight Mode cannot be skipped by WAF custom rules and may still challenge its bots. ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) Perplexity points out that Cloudflare's Free plan allows only one custom IP list, which is not enough to hold the ranges of both bots. ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) Perplexity's AWS WAF recipe pairs an IP set match with a User-Agent string match in one AND statement whose action is Allow. ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) Perplexity tells site owners to refresh their WAF rules from its two JSON endpoints automatically and to treat those files as the source of truth. ([docs.perplexity.ai](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)) ### What our crawl saw Perplexity's perplexitybot.json listed 13 IPv4 prefixes with a creationTime of 2026-10-28 when fetched on 2026-11-05. ([perplexity.ai](https://www.perplexity.ai/perplexitybot.json), dated 2026-10-28) No Web Bot Auth key directory was found at www.perplexity.ai on 2026-11-05; the well-known path returned HTTP 404. ([perplexity.ai](https://www.perplexity.ai/.well-known/http-message-signatures-directory)) ## Products and plans | Product | What it does | Price model | From | As of | | --- | --- | --- | --- | --- | | PerplexityBot | Crawler for the Perplexity search index; Perplexity says it is not used for foundation model training. | No price recorded | | Perplexity-User | Fetcher for pages a user's question needs; Perplexity says it generally ignores robots.txt. | No price recorded | ## Standards | Standard | Status | Evidence | | --- | --- | --- | | RFC 9309 (robots.txt) | Partial | PerplexityBot is managed through robots.txt; Perplexity-User generally ignores robots.txt rules. (docs.perplexity.ai) | | Web Bot Auth | No statement | Not on the crawler page; key directory path 404 on www.perplexity.ai (observed 2026-11-05). | | Published IP list | Supports | perplexitybot.json and perplexity-user.json, one file per bot. (docs.perplexity.ai) | | Reverse DNS check | No statement | Not on the crawler page (read 2026-11-05). | | Cloudflare verified bots | No statement | The page gives a Cloudflare WAF Skip rule recipe but says nothing about verified bot status (read 2026-11-05). | ## Our checks | Check | Result | Method | Date | | --- | --- | --- | --- | | Homepage fetch | HTTP 200 | GET with the crawl user agent after a robots.txt check | 2026-11-05 | | Web Bot Auth key directory | 404 | GET /.well-known/http-message-signatures-directory with the crawl user agent | 2026-11-05 | | perplexitybot.json | 13 IPv4 prefixes, creationTime 2026-10-28 | GET and count prefixes | 2026-11-05 | | perplexity-user.json | 10 IPv4 prefixes, creationTime 2026-10-28 | GET and count prefixes | 2026-11-05 | ## What Perplexity does not do - Perplexity's crawler page documents no identifier for the Comet browser or for Perplexity Computer (read 2026-11-05). - A robots.txt rule does not reliably stop Perplexity-User, which Perplexity says generally ignores robots.txt. ## Related pages on this site - [All crawler operators](https://agentreadycrawlers.com/vendors/by-type/crawler-operator/) 31 profiles - [The full vendor list](https://agentreadycrawlers.com/vendors/) 73 profiles in four types - [Perplexity in the Crawler Index](https://agentreadycrawlers.com/crawlers/operator/perplexity/) 4 rows - [PerplexityBot](https://agentreadycrawlers.com/crawlers/perplexitybot/) token PerplexityBot - [Perplexity-User](https://agentreadycrawlers.com/crawlers/perplexity-user/) token Perplexity-User - [Comet (browser assistant)](https://agentreadycrawlers.com/crawlers/perplexity-comet/) no token - [Perplexity Computer](https://agentreadycrawlers.com/crawlers/perplexity-computer/) no token ## Sources - [perplexity.ai](https://www.perplexity.ai) read 2026-11-05 - [Perplexity Crawlers](https://docs.perplexity.ai/docs/resources/perplexity-crawlers) read 2026-11-05 - [Perplexity perplexitybot.json IP ranges](https://www.perplexity.ai/perplexitybot.json) read 2026-11-05 - [Perplexity perplexity-user.json IP ranges](https://www.perplexity.ai/perplexity-user.json) read 2026-11-05 - [www.perplexity.ai Web Bot Auth key directory path (GET probe)](https://www.perplexity.ai/.well-known/http-message-signatures-directory) read 2026-11-05 Last reviewed 2026-11-05