Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Glossary

Web Bot Auth

In short

An IETF protocol in which bots and agents sign HTTP requests with HTTP Message Signatures and publish their public keys, so sites can verify the operator without trusting IPs or user agents. 8 Index rows sign with Web Bot Auth, and 4 operators in the ledger served a key directory on 2026-11-04.

Definition

An IETF protocol in which bots and agents sign HTTP requests with HTTP Message Signatures and publish their public keys, so sites can verify the operator without trusting IPs or user agents.

Sourced example

Google signs a subset of Google-Agent requests with Web Bot Auth as https://agent.bot.goog. Source, read 2026-11-04.

How it shows up here

8 Index rows sign with Web Bot Auth, and 4 operators in the ledger served a key directory on 2026-11-04.

Previous: Visa Trusted Agent Protocol. Next: Accountable designation. All 41 terms.

Sources

Last reviewed 2026-11-04