Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Crawler operator

Huawei

In short

A site that blocks PetalBot drops out of Petal search entirely, and pages already indexed can linger for several months after the visits stop. The crawl rate follows server capacity, site quality and how often a site changes, and Huawei's webmaster site offers a lookup tool for confirming an address, since no range file is published.

Type
Crawler operator
Homepage
huawei.com
In one line
Huawei runs PetalBot for the Petal search engine, whose index also feeds Huawei Assistant and AI Search; PetalBot follows robots.txt and is verified by reverse DNS under aspiegel.com or an IP lookup tool.
Last read
2026-11-05

What Huawei does for this layer

From the vendor's own pages

Huawei says PetalBot indexes PC and mobile sites for the Petal search engine and for content recommendations in Huawei Assistant and AI Search services. (webmaster.petalsearch.com) Huawei publishes two PetalBot strings: a desktop string, Mozilla/5.0 (compatible;PetalBot;+https://webmaster.petalsearch.com/site/petalbot), and an Android mobile string with the same PetalBot token. (webmaster.petalsearch.com) Huawei says PetalBot complies with the robots protocol and that blocking it makes a site's pages unsearchable in the Petal search engine. (webmaster.petalsearch.com) Huawei tells site owners to verify PetalBot by reverse DNS, where genuine hostnames follow the pattern petalbot.*.aspiegel.com. (webmaster.petalsearch.com)

Huawei says PetalBot adjusts its crawl volume according to server capacity, website quality and website updates. (webmaster.petalsearch.com) Huawei says that after PetalBot stops visiting a site, clearing pages already in the Petal index may take several months. (webmaster.petalsearch.com) The Petal webmaster site offers a Verify PetalBot tool that checks whether a given IP address is a verified PetalBot address. (webmaster.petalsearch.com)

Products and plans

ProductWhat it doesPrice modelFromAs of
PetalBot Crawler for Petal Search, Huawei Assistant and AI Search. No price recorded
Verify PetalBot Lookup tool on the Petal webmaster site that checks an IP address. No price recorded

Standards

StandardStatusEvidence
RFC 9309 (robots.txt) Supports PetalBot complies with the robots protocol. (webmaster.petalsearch.com)
Web Bot Auth No statement Not on the PetalBot page (read 2026-11-05).
Published IP list No statement No range file published; an IP lookup tool instead (read 2026-11-05).
Reverse DNS check Supports Hostnames petalbot.*.aspiegel.com. (webmaster.petalsearch.com)

Our checks

CheckResultMethodDate
Homepage fetch HTTP 200 GET with the crawl user agent after a robots.txt check 2026-11-05

What Huawei does not do

  • Huawei publishes no PetalBot IP range file; checks run through reverse DNS or the webmaster site's lookup tool.
  • Huawei's PetalBot page names no separate token for AI uses, and blocking PetalBot removes a site from Petal search.

Reading notes

Huawei's PetalBot page names aspiegel.com as the reverse DNS domain, while the PetalBot row in the Crawler Index, read on 4 November 2026, gave petalsearch.com. The vendor's own page is the primary source.

Sources

Last reviewed 2026-11-05