--- title: "The Verge agent readiness: the 360 check | Agent Ready Crawlers" description: "The Verge on Ora's agent-readiness scan, read 2026-11-09: 43 of 100, grade D. The door, what agents can read and do, the fixes worth most." canonical: https://agentreadycrawlers.com/blog/theverge-agent-readiness site: "Agent Ready Crawlers" last_updated: 2026-11-09 date: 2026-11-09 section: "360 checks" --- # The Verge agent readiness: the 360 check | Agent Ready Crawlers > The Verge on Ora's agent-readiness scan, read 2026-11-09: 43 of 100, grade D. The door, what agents can read and do, the fixes worth most. Source: https://agentreadycrawlers.com/blog/theverge-agent-readiness Site: Agent Ready Crawlers Last reviewed: 2026-11-09 Agent endpoints: https://agentreadycrawlers.com/llms.txt (index), https://agentreadycrawlers.com/openapi.json (API), https://agentreadycrawlers.com/mcp (MCP server), https://agentreadycrawlers.com/ask (search) Access: https://agentreadycrawlers.com/auth.md (no key). Pricing: https://agentreadycrawlers.com/pricing.md (free). 360 check, 9 November 2026 # The Verge: the 360 agent-readiness check In short The Verge scores 43 of 100 on Ora's agent-readiness scan, a D, read 9 November 2026. At the door, some agents blocked: Claude-User, PerplexityBot. 15 of 71 applicable checks pass; the gap worth most is Public API with reachable endpoints, about 6.3 points by Ora's estimate. Readable pages, an OpenAPI document, and a robots.txt that refuses the assistants. The Verge is Vox Media's technology site, which covers AI crawlers as a beat. On 9 November 2026 its public site scored **43 of 100** on [Ora's](https://ora.ai/?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) agent-readiness scan, a D, in Ora's News & Intelligence category. Ora probes a site the way an agent would: it requests pages with the agents' own user agents, reads robots.txt, looks for the files agents read, and tries the API, MCP and payment surfaces. The check below walks through all 132 of its checks for this one domain, grouped the way an agent meets them. Ora's one-line summary: Turns agents away at the door: robots.txt disallows chatgpt-user, claude-user and perplexity-user, and no public API or MCP server exists. ## The score | Layer | Asks | Points | Share | | --- | --- | --- | --- | | Discovery | Can agents find and recommend it | 9 of 14 | 64% | | Access | Can agents read it | 25 of 48 | 52% | | Usability | Can agents use it | 13 of 49 | 27% | | Payments | Can agents pay | 0 of 0 | none apply | | Total | Grade D | 43 of 100 | 43% | ** 15 checks pass, 10 are partial, 46 fail and 61 do not apply to this site. Every check is defined in [Ora's catalog](https://ora.ai/api/checks?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) (read 2026-11-05); the result for each one is in the groups below, with what Ora saw where it said. ## At the door What the site answers when a request arrives with an agent’s user agent, and what robots.txt says to the crawlers in the Index. Agents named in Ora's results, with their Index rows: [ChatGPT-User](https://agentreadycrawlers.com/crawlers/chatgpt-user/), [OAI-SearchBot](https://agentreadycrawlers.com/crawlers/oai-searchbot/), [Claude-User](https://agentreadycrawlers.com/crawlers/claude-user/), [PerplexityBot](https://agentreadycrawlers.com/crawlers/perplexitybot/), [Perplexity-User](https://agentreadycrawlers.com/crawlers/perplexity-user/). | Check | Result | Points | What Ora saw | | --- | --- | --- | --- | | Not blocked by bot detection | Partial | 1 of 2 | Some agents blocked: Claude-User, PerplexityBot. A normal browser visit from us got in. | | Agent crawler reachability | Partial | 0 of 2 | Some agents are blocked - ChatGPT-User: reachable, OAI-SearchBot: reachable, Claude-User: blocked, PerplexityBot: blocked. A normal browser visit from us got in. | | robots.txt AI crawler policy | Partial | 1 of 2 | AI search crawlers: allowed in 1 of 3 checks (3 crawlers on 1 page). AI training crawlers: allowed in 1 of 2 checks (2 crawlers on 1 page), not scored. robots.txt restricts 2/3 evaluated search page/crawler pairs. | | robots.txt agent-user policy | Fail | 0 of 2 | robots.txt restricts 3/3 evaluated page/crawler pairs. Checked 3 user-triggered agents (chatgpt-user, claude-user, perplexity-user) on 1 public page. Restricted: chatgpt-user at https://theverge.com/ (Disallow: /, | | Content behind auth | Pass | 2 of 2 | All 5 sampled pages are publicly readable (5 with substantive content) | | Homepage not hidden from search | Pass | 1 of 1 | No noindex on the homepage: no robots, googlebot or bingbot meta tag and no X-Robots-Tag header hides it from search. | | Redirect hygiene | Pass | 1 of 1 | Pages agents land on serve real content, not meta-refresh or JavaScript-only redirect stubs that strand non-JS agents | | Web Bot Auth directory | Fail | 0 of 2 | No /.well-known/http-message-signatures-directory found | ## What an agent can read Whether the text is in the first response, what the site publishes for agents, and how it describes itself to them. 2 checks in this group do not apply to this site and are left out. | Check | Result | Points | What Ora saw | | --- | --- | --- | --- | | Content without JavaScript | Partial | 1 of 3 | Homepage: 16554 chars of readable text, but no H1 tag. 5 of 5 sampled inner pages have text without JavaScript. | | Page token budget | Pass | 1 of 1 | All 6 measured pages fit an agent context budget (largest ~4K tokens) | | Sitemap exists | Pass | 2 of 2 | Valid sitemap found at https://www.theverge.com/sitemaps/google_news with 29 entries | | Sitemap freshness (lastmod) | Fail | 0 of 1 | None of the 29 sampled sitemap entries carries a lastmod date. Add (W3C datetime) so agents can prioritize fresh content. | | llms.txt exists | Fail | 0 of 1 | No llms.txt found at /llms.txt or /.well-known/llms.txt. | | llms.txt formatting | Fail | 0 of 2 | No llms.txt found to evaluate quality | | Markdown agent docs | Fail | 0 of 1 | Missing markdown content negotiation. Homepage returned text/html when probed with Accept: text/markdown (expected text/markdown), and none of the well-known root paths (/llms.md, /auth.md, /agents.md, /developers.md, | | Markdown URL fallback | Fail | 0 of 2 | Homepage does not support the .md suffix. Probed https://theverge.com/index.md and it did not return markdown (expected text/markdown content-type or a heading-led non-HTML body). | | Bot-UA markdown serving | Fail | 0 of 1 | No probed bot User-Agent (ChatGPT-User, Claude-User, Perplexity-User) receives markdown when requesting HTML. Optional: detect AI-bot UAs server-side and serve them a markdown representation directly. | | Agent mode view | Fail | 0 of 2 | ?mode=agent returns same content as homepage (no dedicated agent view) | | JSON-LD structured data | Partial | 3 of 4 | JSON-LD Person found - missing description, url, and one of sameAs / jobTitle / worksFor on this entity | | JSON-LD entity linking (sameAs) | Pass | 2 of 2 | sameAs links from your structured data to your official profiles (GitHub, LinkedIn, app stores). They let AIs confirm that every mention of you is actually you. | | Schema type breadth | Fail | 0 of 2 | No extended schema types found - AI can only answer basic questions about this entity | | Metadata completeness | Pass | 2 of 2 | All metadata signals present: canonical URL, lang="en-US", og:image, og:type | | Agent instruction / when-to-use | Fail | 0 of 3 | No agent instruction file with when-to-use guidance found | ## What an agent can do The surfaces an agent acts on: an API it can find and call, an MCP server, an agent card, a command line. 5 checks in this group do not apply to this site and are left out. | Check | Result | Points | What Ora saw | | --- | --- | --- | --- | | Public API with reachable endpoints | Fail | 0 of 7 | No publicly reachable API surface detected (REST and GraphQL both absent or auth-gated) | | Public API/docs linked from homepage | Fail | 0 of 3 | No public API or documentation page linked from homepage | | OpenAPI spec published | Pass | 7 of 7 | OpenAPI spec found at /swagger.json (version: 3.0.0) | | API catalog (RFC 9727) | Fail | 0 of 2 | No /.well-known/api-catalog found (RFC 9727) | | MCP server / manifest | Fail | 0 of 6 | No MCP server or manifest found | | MCP well-known discovery | Fail | 0 of 2 | No MCP server URL discovered | | A2A / agent-card | Fail | 0 of 2 | No A2A agent card found at /.well-known/agent-card.json | | Agent discovery file | Fail | 0 of 2 | No agent discovery file found | | OAuth 2.0 support | Fail | 0 of 5 | No OAuth 2.0 or OpenID Connect support detected | | OAuth Protected Resource metadata (RFC 9728) | Fail | 0 of 2 | No /.well-known/oauth-protected-resource found (RFC 9728) | | auth.md exists | Fail | 0 of 2 | No /auth.md found at site root | | WebMCP support | Fail | 0 of 5 | No WebMCP support detected - no document.modelContext / navigator.modelContext usage found (scanned 41 of 41 same-origin script bundle(s) for modelContext registrations) | | NLWeb /ask endpoint | Fail | 0 of 1 | No NLWeb-conformant /ask endpoint found via GET or POST | | JSON error responses | Fail | 0 of 4 | API does not return JSON error responses (or no API detected) | ## Found and recommended Whether an assistant can find the company by name, confirm what it is, and see it in the registries agents read. 3 checks in this group do not apply to this site and are left out. | Check | Result | Points | What Ora saw | | --- | --- | --- | --- | | Brand name discoverability | Pass | 3 of 3 | theverge.com ranks #2 for "The Verge tech news" and #4 for "The Verge technology" (search term updated) | | Wikipedia / Wikidata entity presence | Pass | 4 of 4 | Wikipedia "The Verge" and Wikidata Q2155442 both verified - domain confirmed on both sources | | Developer resource discoverability | Fail | 0 of 3 | In a search for your API docs, none of the 10 results is your official docs | | ChatGPT app listed | Fail | 0 of 2 | Not found in ChatGPT app directory | | Listed on skills.sh | Fail | 0 of 1 | No official skills self-published on skills.sh (18 community skills with potential relation to this domain found, but only self-published official skills count) | | ARD discovery | Fail | 0 of 1 | No /.well-known/ard.json, no same-site rel="ard" link, and no legacy /.well-known/ai-catalog.json | ## The fixes worth most points Ora attaches an estimate to each failing or partial check: how many points the site would gain by passing it. These are the 5 with the largest estimates for The Verge, with what Ora observed. The wording of each fix is Ora's job; the links go to the check in its catalog. Public API with reachable endpoints Fail, 0 of 7, about 6.3 points to gain. Ora saw: No publicly reachable API surface detected (REST and GraphQL both absent or auth-gated) MCP server / manifest Fail, 0 of 6, about 5.4 points to gain. Ora saw: No MCP server or manifest found Developer resource discoverability Fail, 0 of 3, about 4.8 points to gain. Ora saw: In a search for your API docs, none of the 10 results is your official docs OAuth 2.0 support Fail, 0 of 5, about 4.5 points to gain. Ora saw: No OAuth 2.0 or OpenID Connect support detected Scoped permissions Fail, 0 of 5, about 4.5 points to gain. Ora saw: No declared OAuth scopes, security schemes, or scoped-permission documentation found ## Our read The Verge is a technology publication whose front door treats the agents it covers unevenly: ChatGPT-User and OAI-SearchBot reached the home page while Claude-User and PerplexityBot were blocked, and robots.txt restricts all three user-triggered agents on every check Ora ran. Once inside, the site is generous. The home page returned 16,554 characters of text without JavaScript, five of five sampled articles were readable, and Ora found an OpenAPI 3.0 document at /swagger.json, a rare thing on a news site. The gaps are the usual publisher set: no public API behind that document, no MCP server, no OAuth, and a Person entity in the JSON-LD missing its description. A D at 43 for a site with this much readable text is a robots.txt problem first; the file says one thing to Google and another to the reader's assistant. ## Check your own door The scan above is free and needs no account: enter a domain at [ora.ai](https://ora.ai/?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) and read the same four layers for your own site, then compare the door with the [Block Rate Report](https://agentreadycrawlers.com/block-rates/) for sites of your rank and kind. Our [guide to testing whether agents can reach your site](https://agentreadycrawlers.com/guides/test-whether-ai-agents-can-reach-your-site/) has the curl commands for a manual check, and the [robots.txt builder](https://agentreadycrawlers.com/robots-builder/) writes a policy file from the Index. Payments did not apply to this site; a site that sells would read that layer against the [vendors at the door](https://agentreadycrawlers.com/vendors/ranking/). [Scan your own domain on Ora](https://ora.ai/?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) [All 360 checks](https://agentreadycrawlers.com/blog/). One company a day; corrections to [the corrections page](https://agentreadycrawlers.com/corrections/). ** The check we recommend [Ora](https://ora.ai/?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) scans a public URL: 16 Discovery checks and 42 Access checks, read from its catalog on 2026-11-05. Its Journey sends a real agent to the site. [How to run it](https://agentreadycrawlers.com/guides/test-whether-ai-agents-can-reach-your-site/). **Limits.** Ora reads public URLs only, and what it records as blocked comes from its own network. It says the same scan run twice converges within a couple of points ([Ora's Deep Scan post](https://ora.ai/blog/introducing-deep-scan?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check)), and the pages we read give no repeat-scan figures. Its fixing product is behind a waitlist ([Ora's home page](https://ora.ai/?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check)). ## Sources - [Ora's scan of theverge.com, GET /api/score (vendor)](https://ora.ai/api/score/theverge.com?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) read 2026-11-09 - [Ora's check catalog, GET /api/checks (vendor)](https://ora.ai/api/checks?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) read 2026-11-05 - [The Verge home page](https://www.theverge.com/) read 2026-11-09 - [How to test whether AI agents can reach your site (this site)](https://agentreadycrawlers.com/guides/test-whether-ai-agents-can-reach-your-site/) read 2026-11-04 - [Ora: Deep Scan v2 (vendor)](https://ora.ai/blog/introducing-deep-scan?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) read 2026-11-05 - [Ora home page, waitlist (vendor)](https://ora.ai/?utm_source=agentreadycrawlers.com&utm_medium=referral&utm_campaign=check) read 2026-11-05 Last reviewed 2026-11-09