Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Crawler operator

Block

In short

Block wrote goose, an open-source agent that runs on the user's own machine and that Cloudflare listed among its first signed agents in 2025. Its code now lives under the Agentic AI Foundation, part of the Linux Foundation, rather than at Block. Neither Block nor the goose docs offer a site owner any way to verify a visit.

Type
Crawler operator
Homepage
block.xyz
In one line
Company that built goose, an open-source local AI agent Cloudflare classed as a signed agent in 2025; Block has since donated goose to the Linux Foundation's Agentic AI Foundation and publishes no key directory for it.
Last read
2026-11-05

What Block does for this layer

From the vendor's own pages

Block has donated goose to the Agentic AI Foundation at the Linux Foundation, alongside Anthropic's MCP and OpenAI's AGENTS.md, and the goose blog post of 2026-04-07 says the repository moved from block/goose to aaif-goose/goose. (goose-docs.ai, dated 2026-04-07) goose runs on the user's machine as a desktop app for macOS, Linux and Windows, a CLI and an API, and is built in Rust, per its documentation home page. (goose-docs.ai) goose works with 15+ model providers, including Anthropic, OpenAI, Google, Ollama, OpenRouter, Azure and Bedrock, and can use existing Claude, ChatGPT or Gemini subscriptions through ACP. (goose-docs.ai) The goose repository is Apache-2.0 licensed and showed 56.2k stars on 2026-11-05. (github.com)

Reported by a third party

Cloudflare listed goose from Block in its first cohort of signed agents on 2025-08-28, recognised through Web Bot Auth signatures. (blog.cloudflare.com, dated 2025-08-28) In Cloudflare's signed-agents post, a Block staff software engineer says goose was designed to run locally and that Web Bot Auth lets third-party sites trust goose as it acts on users' behalf. (blog.cloudflare.com, dated 2025-08-28)

What our crawl saw

The goose documentation index read on 2026-11-05 has no page on Web Bot Auth, request signing or a user agent for site owners. (goose-docs.ai) A GET for block.xyz/.well-known/http-message-signatures-directory returned HTTP 404 on 2026-11-05. (block.xyz)

Products and plans

ProductWhat it doesPrice modelFromAs of
goose Open-source general-purpose AI agent (desktop, CLI, API), now governed by the Agentic AI Foundation. No price recorded

Standards

StandardStatusEvidence
RFC 9309 (robots.txt) No statement No robots.txt statement in the goose docs index (2026-11-05).
Web Bot Auth Announced Cloudflare's 2025-08-28 post names goose as a signed agent; no goose or Block page documents the signing. (blog.cloudflare.com)
Cloudflare verified bots Announced Named in Cloudflare's first signed-agent cohort. (blog.cloudflare.com)

Our checks

CheckResultMethodDate
Homepage fetch HTTP 200 GET with the crawl user agent after a robots.txt check 2026-11-05
Web Bot Auth key directory HTTP 404, text/html GET https://block.xyz/.well-known/http-message-signatures-directory with the crawl user agent after a robots.txt check 2026-11-05

What Block does not do

  • Block publishes no key directory, user agent string or verification page for goose (checked 2026-11-05), so its signed-agent status rests on Cloudflare's 2025 announcement.
  • Block no longer hosts goose's code; the repository now sits in the aaif-goose organisation under the Agentic AI Foundation.

Sources

Last reviewed 2026-11-05