---
title: "xAI: crawler operator profile | Agent Ready Crawlers"
description: "xAI: crawler operator profile covering Grok Bot and Web Search tool (xAI API). Sourced facts, standards, dated prices and limits, read 2026-11-05."
canonical: https://agentreadycrawlers.com/vendors/xai
site: "Agent Ready Crawlers"
last_updated: 2026-11-05
section: "Vendors"
---

# xAI: crawler operator profile | Agent Ready Crawlers

> xAI: crawler operator profile covering Grok Bot and Web Search tool (xAI API). Sourced facts, standards, dated prices and limits, read 2026-11-05.

Source: https://agentreadycrawlers.com/vendors/xai
Site: Agent Ready Crawlers
Last reviewed: 2026-11-05

Crawler operator

# xAI

In short

xAI gives Grok Bot an always-on cloud computer of its own, shared by an account's other Bots, and lets it hand control back to the user at a CAPTCHA or a passkey. A site learns little about it: neither x.ai nor grok.com serves a key directory, and a third party saw signing end in mid-October.

Type

[Crawler operator](https://agentreadycrawlers.com/vendors/by-type/crawler-operator/)

Homepage

[x.ai](https://x.ai)

In one line

xAI operates Grok Bot, an agent product in which each Bot works on a persistent cloud computer with a browser; xAI's docs give site owners no user agent, IP list or signing keys for that traffic.

Last read

2026-11-05

## What xAI does for this layer

### From the vendor's own pages

xAI's docs say each Grok Bot works on a persistent cloud computer with a browser, filesystem and terminal, and keeps working while the user's laptop is closed. ([docs.x.ai](https://docs.x.ai/grok-bot/overview)) xAI includes Grok Bot with the Premium, Super and Ultra tiers of The X Subscription and with paid individual plans on legacy Cursor plans. ([docs.x.ai](https://docs.x.ai/grok-bot/overview)) xAI says all Bots on one account share a single cloud computer, including browser cookies, signed-in sessions and files. ([docs.x.ai](https://docs.x.ai/grok-bot/computer-and-apps)) Grok Bot hands the computer back to the user for steps it cannot finish, which xAI lists as single sign-on, passkeys, approvals on another device and "A CAPTCHA it couldn't get past". ([docs.x.ai](https://docs.x.ai/grok-bot/computer-and-apps))

Grok Bot can pay at checkout with a card saved in its Vault, a card entered in a Secure Form, or a connected wallet such as Link or X Money. ([docs.x.ai](https://docs.x.ai/grok-bot/computer-and-apps)) xAI's API web\_search tool lets developers limit browsing with allowed\_domains or excluded\_domains, up to five domains each. ([docs.x.ai](https://docs.x.ai/developers/tools/web-search))

### Reported by a third party

Verisoul reported on 2026-11-03 that Grok Bot stopped signing requests with Web Bot Auth in mid-October 2026 and routes its cloud browser traffic through a reverse proxy on the user's own device, so sites see the user's IP. ([verisoul.ai](https://www.verisoul.ai/agents/grok-bot), dated 2026-11-03)

### What our crawl saw

The Web Bot Auth key directory path returned HTTP 404 on both x.ai and grok.com on 2026-11-05. ([x.ai](https://x.ai/.well-known/http-message-signatures-directory))

## Products and plans

| Product | What it does | Price model | From | As of |
| --- | --- | --- | --- | --- |
| Grok Bot | Always-on agent product; each Bot browses and acts from a persistent cloud computer. | No price recorded |
| Web Search tool (xAI API) | Server-side search and browsing tool for API callers, with domain allow and exclude filters. | No price recorded |

## Standards

| Standard | Status | Evidence |
| --- | --- | --- |
| RFC 9309 (robots.txt) | No statement | xAI's docs index lists no crawler or robots.txt page (read 2026-11-05). |
| Web Bot Auth | No statement | No key directory on x.ai or grok.com (observed 404, 2026-11-05). |
| Published IP list | No statement | No IP list in xAI's docs (read 2026-11-05). |

## Our checks

| Check | Result | Method | Date |
| --- | --- | --- | --- |
| Homepage fetch | HTTP 403 | GET with the crawl user agent after a robots.txt check | 2026-11-05 |
| Web Bot Auth key directory | 404 on x.ai and grok.com | GET /.well-known/http-message-signatures-directory with the crawl user agent | 2026-11-05 |

## What xAI does not do

-   xAI publishes no robots.txt token for any crawler or fetcher in its docs index.

## Related pages on this site

-   [All crawler operators](https://agentreadycrawlers.com/vendors/by-type/crawler-operator/) 31 profiles
-   [The full vendor list](https://agentreadycrawlers.com/vendors/) 73 profiles in four types
-   [Grok Bot (agent)](https://agentreadycrawlers.com/crawlers/grok-bot/) no token

## Sources

-   [x.ai](https://x.ai) read 2026-11-05
-   [Grok Bot overview (xAI docs)](https://docs.x.ai/grok-bot/overview) read 2026-11-05
-   [Grok Bot: Use the computer and apps (xAI docs)](https://docs.x.ai/grok-bot/computer-and-apps) read 2026-11-05
-   [xAI API Tools: Web Search](https://docs.x.ai/developers/tools/web-search) read 2026-11-05
-   [How to detect Grok Bot (Verisoul Agent Atlas)](https://www.verisoul.ai/agents/grok-bot) read 2026-11-05
-   [x.ai Web Bot Auth key directory path (GET probe)](https://x.ai/.well-known/http-message-signatures-directory) read 2026-11-05

Last reviewed 2026-11-05