--- title: "Storebot-Google: user agent, token, block rate | Agent Ready Crawlers" description: "Storebot-Google from Google: exact user-agent string, robots.txt token Storebot-Google, search index purpose and how to verify it. Read 2026-11-04." canonical: https://agentreadycrawlers.com/crawlers/storebot-google site: "Agent Ready Crawlers" last_updated: 2026-11-04 section: "Crawlers" --- # Storebot-Google: user agent, token, block rate | Agent Ready Crawlers > Storebot-Google from Google: exact user-agent string, robots.txt token Storebot-Google, search index purpose and how to verify it. Read 2026-11-04. Source: https://agentreadycrawlers.com/crawlers/storebot-google Site: Agent Ready Crawlers Last reviewed: 2026-11-04 Search index crawler from Google # Storebot-Google In short Storebot-Google is Google's search index crawler. The token to write in robots.txt is Storebot-Google. Verify it by reverse DNS, then a forward lookup. The crawl of 2026-11-04 did not measure Storebot-Google. Relevant to commerce sites: blocking it removes Shopping surfaces. Its user-agent entry: Mozilla/5.0 (X11; Linux x86\_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36. Storebot-Google obeys robots.txt, per Google. Operator [Google](https://agentreadycrawlers.com/crawlers/operator/google/) Vendor profile: [Google](https://agentreadycrawlers.com/vendors/google/) Purpose Search index [All search index rows](https://agentreadycrawlers.com/crawlers/purpose/search-index/). Builds a search index that sends visitors back. Refusing it removes pages from that product. User-agent string `Mozilla/5.0 (X11; Linux x86_64; Storebot-Google/1.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Safari/537.36` robots.txt token `Storebot-Google` Matched case-insensitively (RFC 9309). The token and the user-agent string are different things. Respects robots.txt Yes, per operator Common crawler: obeys robots.txt, per Google. Preferences affect all surfaces of Google Shopping. [Source](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers), read 2026-11-04. Verification Reverse DNS Reverse-resolve the source address, then forward-resolve the name and require the same address. [Documentation](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests), read 2026-11-04. IP ranges [developers.google.com/static/search/apis/ipranges/googlebot.json](https://developers.google.com/static/search/apis/ipranges/googlebot.json) Rate behaviour Not published separately. Block rate Not measured: the crawl checked 31 AI and search tokens, and `Storebot-Google` is not one of them. Documentation [developers.google.com](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers), read 2026-11-04. Note. Relevant to commerce sites: blocking it removes Shopping surfaces. ## Allow and refuse lines ### To allow Storebot-Google User-agent: Storebot-Google Allow: / ### To refuse Storebot-Google User-agent: Storebot-Google Disallow: / Refusing or allowing in robots.txt states a preference. Enforcement of a refusal belongs in your WAF; see how to verify the identity above. ## Google in the Verification Ledger Five IP lists, reverse DNS for every class, and an experimental key directory for Google-Agent. Nothing for auto browse in the user's Chrome. [All operators](https://agentreadycrawlers.com/verification/). IP lists - [developers.google.com/static/search/apis/ipranges/googlebot.json](https://developers.google.com/static/search/apis/ipranges/googlebot.json) - [developers.google.com/static/search/apis/ipranges/special-crawlers.json](https://developers.google.com/static/search/apis/ipranges/special-crawlers.json) - [developers.google.com/static/search/apis/ipranges/user-triggered-fetchers.json](https://developers.google.com/static/search/apis/ipranges/user-triggered-fetchers.json) - [developers.google.com/static/search/apis/ipranges/user-triggered-fetchers-google.json](https://developers.google.com/static/search/apis/ipranges/user-triggered-fetchers-google.json) - [developers.google.com/static/search/apis/ipranges/user-triggered-agents.json](https://developers.google.com/static/search/apis/ipranges/user-triggered-agents.json) Reverse DNS crawl-\*.googlebot.com, geo-crawl-\*.geo.googlebot.com, rate-limited-proxy-\*.google.com, \*.gae.googleusercontent.com, google-proxy-\*.google.com Key directory https://agent.bot.goog/.well-known/http-message-signatures-directory (Google-Agent, subset of requests, experimental) Cloudflare not checked Visa TAP registry no public operator-level registration found (checked 2026-11-04) Mastercard Agent Pay no public operator-level registration found (checked 2026-11-04) ## Other Google rows - [Googlebot](https://agentreadycrawlers.com/crawlers/googlebot/) Search index token Googlebot - [Google-Extended (control token)](https://agentreadycrawlers.com/crawlers/google-extended/) Training token Google-Extended - [Google-CloudVertexBot](https://agentreadycrawlers.com/crawlers/google-cloudvertexbot/) Other token Google-CloudVertexBot - [GoogleOther](https://agentreadycrawlers.com/crawlers/googleother/) Other token GoogleOther - [Google-Agent](https://agentreadycrawlers.com/crawlers/google-agent/) Task agent no token - [Gemini in Chrome (auto browse)](https://agentreadycrawlers.com/crawlers/gemini-in-chrome/) Task agent no token - [Google-GeminiNotebook (was Google-NotebookLM)](https://agentreadycrawlers.com/crawlers/google-gemininotebook/) Live answer fetch no token ## Sources - [Google common crawlers](https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers) read 2026-11-04 - [Verify requests from Google](https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests) read 2026-11-04 - [developers.google.com: googlebot (JSON)](https://developers.google.com/static/search/apis/ipranges/googlebot.json) read 2026-11-04 Last reviewed 2026-11-04