--- title: "Parallel Web Systems: crawler operator profile | Agent Ready Crawlers" description: "Parallel Web Systems: crawler operator profile covering Parallel Search, Extract and Task APIs. Standards, dated prices and limits, read 2026-11-05." canonical: https://agentreadycrawlers.com/vendors/parallel site: "Agent Ready Crawlers" last_updated: 2026-11-05 section: "Vendors" --- # Parallel Web Systems: crawler operator profile | Agent Ready Crawlers > Parallel Web Systems: crawler operator profile covering Parallel Search, Extract and Task APIs. Standards, dated prices and limits, read 2026-11-05. Source: https://agentreadycrawlers.com/vendors/parallel Site: Agent Ready Crawlers Last reviewed: 2026-11-05 Crawler operator # Parallel Web Systems In short Parallel Web Systems feeds its search and extraction APIs from two signed identities, documented from 24 October 2026. ShapBot builds the index and has a robots.txt token and an IP list; Shap-User fetches pages at a user's direction and offers no robots.txt control. Search requests are priced from USD 0.001. Type [Crawler operator](https://agentreadycrawlers.com/vendors/by-type/crawler-operator/) Homepage [parallel.ai](https://parallel.ai) In one line Web search and research API company for AI agents that crawls as ShapBot and fetches for users as Shap-User, both signed with Web Bot Auth, with an IP list for ShapBot and a robots.txt token for the crawler only. Last read 2026-11-05 ## What Parallel Web Systems does for this layer ### From the vendor's own pages Parallel Web Systems documents two identities on a bots page effective 2026-10-24: ShapBot, which builds Parallel's search index, and Shap-User, which fetches pages at users' direction. ([parallel.ai](https://parallel.ai/parallel-web-systems-bots), dated 2026-10-24) ShapBot's full user agent is "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0", and Parallel tells site owners to manage it with a User-agent: ShapBot group in robots.txt. ([parallel.ai](https://parallel.ai/parallel-web-systems-bots)) Shap-User sends a user agent ending "compatible; Shap-User/0.1.0", and Parallel says Shap-User "is intended to provide visibility rather than to be managed through automated controls". ([parallel.ai](https://parallel.ai/parallel-web-systems-bots)) Parallel signs both ShapBot and Shap-User requests with Web Bot Auth and publishes keys at shapbot.parallel.ai and shap-user.parallel.ai under /.well-known/http-message-signatures-directory. ([parallel.ai](https://parallel.ai/parallel-web-systems-bots)) Parallel warns that restricting ShapBot "prevents our system from indexing your content", which it says may reduce a site's discoverability through its web APIs and search features. ([parallel.ai](https://parallel.ai/parallel-web-systems-bots)) Parallel invites content owners with questions about its index to explore joining "Index by Parallel" through a dedicated contact address. ([parallel.ai](https://parallel.ai/parallel-web-systems-bots)) Parallel's pricing page allows up to 5,000 requests a month free and prices the Search API at USD 0.001 to USD 0.005 for 10 results and the Extract API at USD 0.001 per request. ([parallel.ai](https://parallel.ai/pricing)) ### What our crawl saw Parallel's ShapBot IP list at docs.parallel.ai/resources/shapbot.json had a creationTime of 2026-07-14 and 10 single-address IPv4 prefixes when read on 2026-11-05. ([docs.parallel.ai](https://docs.parallel.ai/resources/shapbot.json)) The ShapBot and Shap-User key directories both returned HTTP 200 with content type application/http-message-signatures-directory+json and two Ed25519 keys each on 2026-11-05. ([shapbot.parallel.ai](https://shapbot.parallel.ai/.well-known/http-message-signatures-directory)) ## Products and plans | Product | What it does | Price model | From | As of | | --- | --- | --- | --- | --- | | Parallel Search, Extract and Task APIs | Web search, page extraction and research tasks for agents, built on the ShapBot index. | pay per request Search API low end per request for 10 results; 5,000 free requests a month. | USD 0.001 | 2026-11-05 | ## Standards | Standard | Status | Evidence | | --- | --- | --- | | RFC 9309 (robots.txt) | Supports | ShapBot is managed with a User-agent: ShapBot robots.txt group. (parallel.ai) | | Web Bot Auth | Supports | ShapBot and Shap-User sign requests; key directories on two hosts. (parallel.ai) | | Published IP list | Partial | IP list published for ShapBot only, not for Shap-User. (docs.parallel.ai) | ## Our checks | Check | Result | Method | Date | | --- | --- | --- | --- | | Homepage fetch | HTTP 200 | GET with the crawl user agent after a robots.txt check | 2026-11-05 | | Web Bot Auth key directory | HTTP 200, application/http-message-signatures-directory+json, 2 Ed25519 keys | GET https://shapbot.parallel.ai/.well-known/http-message-signatures-directory with the crawl user agent after a robots.txt check | 2026-11-05 | | Web Bot Auth key directory | HTTP 200, application/http-message-signatures-directory+json, 2 Ed25519 keys | GET https://shap-user.parallel.ai/.well-known/http-message-signatures-directory with the crawl user agent after a robots.txt check | 2026-11-05 | | ShapBot IP list | creationTime 2026-07-14, 10 IPv4 /32 prefixes | GET https://docs.parallel.ai/resources/shapbot.json with the crawl user agent | 2026-11-05 | ## What Parallel Web Systems does not do - Parallel gives Shap-User no robots.txt control; its bots page says user-initiated fetches are for visibility, not automated management. ## Reading notes ShapBot and Shap-User from Parallel are not in the [Crawler Index](https://agentreadycrawlers.com/crawlers/) yet. ## Related pages on this site - [All crawler operators](https://agentreadycrawlers.com/vendors/by-type/crawler-operator/) 31 profiles - [The full vendor list](https://agentreadycrawlers.com/vendors/) 73 profiles in four types ## Sources - [parallel.ai](https://parallel.ai) read 2026-11-05 - [Overview of Parallel Web Systems' Bots | Parallel](https://parallel.ai/parallel-web-systems-bots) read 2026-11-05 - [ShapBot IP list (shapbot.json)](https://docs.parallel.ai/resources/shapbot.json) read 2026-11-05 - [ShapBot key directory](https://shapbot.parallel.ai/.well-known/http-message-signatures-directory) read 2026-11-05 - [Parallel Pricing | Parallel](https://parallel.ai/pricing) read 2026-11-05 - [Shap-User key directory](https://shap-user.parallel.ai/.well-known/http-message-signatures-directory) read 2026-11-05 Last reviewed 2026-11-05