Crawler operator
Kernel
In short
Kernel treats signing as a complement to its stealth mode and residential proxies, not a guarantee of access, and grants it by request to Startup and Enterprise customers. Its two signed identities, Kernel Agent and KernelSearchBot, each have a separate key directory, and the free tier carries USD 5 of monthly credits.
- Type
- Crawler operator
- Homepage
- kernel.sh
- In one line
- Cloud browser company for AI agents that runs two signed identities, Kernel Agent for user-directed browsing and KernelSearchBot for search crawling, each with its own Web Bot Auth key directory so sites can treat them separately.
- Last read
- 2026-11-05
What Kernel does for this layer
From the vendor's own pages
Kernel Search identifies itself with the KernelSearchBot user agent token and follows robots.txt directives for that token, including crawl-delay. (kernel.sh) Kernel says Kernel Agent is not an automatic crawler and signs its requests with Web Bot Auth instead of sending a dedicated crawler user agent token. (kernel.sh) Kernel serves a separate key directory for each identity so that site owners can allow search indexing while blocking user-directed agents, or the reverse. (kernel.sh) Kernel's Web Bot Auth token is an opt-in feature enabled selectively on request for Startup and Enterprise plan customers, and being on an eligible plan does not switch it on. (kernel.sh)
Kernel's docs say Web Bot Auth complements its stealth mode and proxies, which deal with browser and network signals, and does not guarantee access to every site. (kernel.sh) Kernel's pricing page lists Free at USD 0 with USD 5 of monthly credits and 5 concurrent browsers, Hobbyist at USD 30 a month and Start-up at USD 200 a month with 150 concurrent browsers, plus usage at USD 0.0000166667 per GB-second. (kernel.sh) Kernel says its usage rates include proxy traffic and that free tiers include stealth mode, which attaches a Kernel-managed residential proxy and a captcha auto-solver to each browser. (kernel.sh)
Claims the vendor makes about itself
Kernel says it is listed in Cloudflare's bots and agents directory and in Vercel's public directory at bots.fyi. (kernel.sh)
What our crawl saw
Both Kernel key directories, www.kernel.sh and search.bot.kernel.sh, returned HTTP 200 with content type application/http-message-signatures-directory+json and one Ed25519 key each on 2026-11-05. (kernel.sh)
Products and plans
| Product | What it does | Price model | From | As of |
|---|---|---|---|---|
| Kernel cloud browsers | Isolated Chromium for agents with stealth, proxies, managed auth and opt-in Web Bot Auth. | subscription plus usage | USD 0 | 2026-11-05 |
| Kernel Search (KernelSearchBot) | Crawler that builds search indexes and retrieval databases; signed and robots.txt-compliant. | |||
Standards
| Standard | Status | Evidence |
|---|---|---|
| RFC 9309 (robots.txt) | Supports | KernelSearchBot follows robots.txt for its token, crawl-delay included. (kernel.sh) |
| Web Bot Auth | Supports | Kernel Agent and Kernel Search sign with Web Bot Auth; key directories published per identity. (kernel.sh) |
| Published IP list | No statement | No IP list on the bots page (2026-11-05). |
| Cloudflare verified bots | Supports | Kernel says it is listed in Cloudflare's bots and agents directory. (kernel.sh) |
Our checks
| Check | Result | Method | Date |
|---|---|---|---|
| Homepage fetch | HTTP 200 | GET with the crawl user agent after a robots.txt check | 2026-11-05 |
| Web Bot Auth key directory | HTTP 200, application/http-message-signatures-directory+json, 1 Ed25519 key | GET https://www.kernel.sh/.well-known/http-message-signatures-directory with the crawl user agent after a robots.txt check | 2026-11-05 |
| Web Bot Auth key directory | HTTP 200, application/http-message-signatures-directory+json, 1 Ed25519 key | GET https://search.bot.kernel.sh/.well-known/http-message-signatures-directory with the crawl user agent after a robots.txt check | 2026-11-05 |
What Kernel does not do
- Kernel does not sign customer sessions with its identity unless it approves a request on an eligible plan; other sessions rely on stealth mode and residential proxies.
- Kernel publishes no IP list for KernelSearchBot or Kernel Agent (bots page read 2026-11-05).
Reading notes
KernelSearchBot and Kernel Agent are not rows in the Crawler Index yet.
Related pages on this site
Sources
Last reviewed 2026-11-05