Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Licensing intermediary

FetchRight

In short

In Peek-Then-Pay a publisher signals participation in /.well-known/peek.json, naming the licence server and terms, and an agent answers a priced preview by buying a JWT licence for one intent, such as quote or embed. Edge enforcers check those licences on their own, and FetchRight claims previews cut AI compute by over 90%.

Type
Licensing intermediary
Homepage
fetchright.ai
In one line
Licensing service behind the open Peek-Then-Pay protocol: a site publishes /.well-known/peek.json, AI agents get a priced HTTP 203 preview, buy a JWT license per intent and present it to an edge enforcer.
Last read
2026-11-05

What FetchRight does for this layer

From the vendor's own pages

FetchRight describes Peek-Then-Pay as an open protocol that separates discovery from access, so AI agents can evaluate content through structured previews without unrestricted retrieval. (fetchright.ai) Under Peek-Then-Pay a publisher signals participation with a manifest at /.well-known/peek.json that names the license server, the terms URL, the supported intents and rate limits per IP. (peekthenpay.org) Peek-Then-Pay answers an AI agent's content request with HTTP 203 Non-Authoritative Information carrying a publisher-written preview plus pricing. (github.com) The Peek-Then-Pay specification chooses HTTP 203 over 402 because, in its words, most AI systems process 203 responses normally "unlike 402 which is often blocked". (github.com)

Peek-Then-Pay defines six standard intents, peek, read, summarize, quote, embed and analyze, and agents buy a JWT license for a specific intent from the license server in the manifest. (peekthenpay.org) Peek-Then-Pay edge enforcers validate assertion-only JWTs and manage budgets locally without calling the license server, and both sides report usage asynchronously for billing and disputes. (github.com) The example peek.json on peekthenpay.org names https://api.fetchright.ai/v1/license as license issuer and lists /robots.txt and /sitemap.xml as paths that bypass enforcement. (peekthenpay.org) The peek.json specification is dedicated to the public domain under CC0 1.0 and published as the npm package @peekthenpay/peek-json-spec. (github.com)

Peek-Then-Pay's sample pricing schema prices intents per request or per 1,000 tokens in US cents, and marks train and index uses as requiring a contract. (peekthenpay.org)

Claims the vendor makes about itself

FetchRight claims that structured peek responses cut AI compute by more than 90% compared with fetching raw HTML. (fetchright.ai)

Products and plans

ProductWhat it doesPrice modelFromAs of
Peek-Then-Pay Open protocol (CC0) for manifest discovery, HTTP 203 previews, intent-scoped JWT licenses and edge enforcement. open specification CC0 1.0 USD 0 2026-11-05
FetchRight platform Peek and Pay services for publishers, affiliates, partner networks and brands; acts as a license server. sales-led homepage offers "Start a Conversation" only Not published 2026-11-05

Standards

StandardStatusEvidence
RFC 9309 (robots.txt) No statement Peek-Then-Pay keeps /robots.txt reachable in its example manifest but does not build on robots.txt directives (checked 2026-11-05).
Web Bot Auth No statement Agents prove entitlement with license JWTs; the spec page does not reference Web Bot Auth (checked 2026-11-05).

Our checks

CheckResultMethodDate
Homepage fetch HTTP 200 GET with the crawl user agent after a robots.txt check 2026-11-05

What FetchRight does not do

  • Peek-Then-Pay does not stop an agent that never reads peek.json; enforcement depends on the publisher running an edge enforcer.
  • FetchRight publishes no price list or fee for its license server (homepage read 2026-11-05).

Reading notes

peekthenpay.org links fetchright.ai as the company behind the protocol, and its example manifest names api.fetchright.ai as the licence issuer.

Sources

Last reviewed 2026-11-05