Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

News from 2026-11-02

Signed-Agent-Metadata header proposed for Web Bot Auth

In short

2 November 2026: Signed-Agent-Metadata header proposed for Web Bot Auth. draft-vilgalys-webbotauth-signed-agent-metadata-00 proposes an HTTP header that carries signed metadata about the agent alongside a Web Bot Auth signature. For a site owner nothing changes yet: a proposed header is not something any operator sends today, so no rule needs to change.

draft-vilgalys-webbotauth-signed-agent-metadata-00 proposes an HTTP header that carries signed metadata about the agent alongside a Web Bot Auth signature.

Primary source datatracker.ietf.org/wg/webbotauth/documents/, read 2026-11-04.

All news.

Sources

Last reviewed 2026-11-04