Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Glossary

ai-input

In short

The Content Signals value for using content as live input to an AI answer, such as grounding or retrieval. Leaving it blank states no preference. The builder sets it from the Live answer fetch choice: allow gives ai-input=yes and refuse gives ai-input=no. Example: Cloudflare serves no ai-input value for managed robots.txt customers because it does not know their preference.

Definition

The Content Signals value for using content as live input to an AI answer, such as grounding or retrieval. Leaving it blank states no preference.

Sourced example

Cloudflare serves no ai-input value for managed robots.txt customers because it does not know their preference. Source, read 2026-11-04.

How it shows up here

The builder sets it from the Live answer fetch choice: allow gives ai-input=yes and refuse gives ai-input=no.

Previous: Agentic browser. Next: AIPREF training signal. All 41 terms.

Sources

Last reviewed 2026-11-04