Agent Ready Crawlers

The field guide to every AI crawler and agent at the door, and the vendors who decide who gets in.

Crawler operator

Hive

In short

Hive changed its name to Deploy on 26 October 2026 and now sells hosted AI models. Its ImageSift unit crawls public images for a reverse image search index, keeping each picture's alt text along with the surrounding text, and accepts opt-outs by email only; no IP list or signature keys exist to confirm a visit.

Type
Crawler operator
Homepage
deploy.com
In one line
AI model company, renamed from Hive to Deploy in October 2026, whose ImageSift unit runs ImagesiftBot to collect public images and page text for a reverse image search index.
Last read
2026-11-05

What Hive does for this layer

From the vendor's own pages

ImageSift's about page says ImageSiftBot scrapes the internet for publicly available images to support Hive's web intelligence products and sends the user agent "Mozilla/5.0 (compatible; ImagesiftBot; +imagesift.com)". (imagesift.com) ImageSift respects robots.txt groups addressed to ImagesiftBot, and where no group names ImagesiftBot but one names Googlebot, ImagesiftBot follows the Googlebot rules instead of the * group. (imagesift.com) ImagesiftBot reads Crawl-delay as the minimum seconds between the starts of consecutive requests, splitting each day into intervals of that length and sending at most one request per interval. (imagesift.com) Along with each image, ImageSiftBot saves the host URL, the text on the page and the image's alt text, which go into an index used for similar-image search. (imagesift.com)

ImageSift's page offers opt-out by email to ImageSift support and names no form, IP list or verification method. (imagesift.com) imagesift.com itself serves a reverse image search box that takes an uploaded file or an image URL. (imagesift.com) Hive announced on 2026-10-26 that it is now called Deploy and is broadening into an enterprise AI inference platform serving open-source and proprietary models. (thehive.ai, dated 2026-10-26) Deploy's pricing page offers pay-as-you-go billing for developers and custom Enterprise pricing, with hosted models such as DeepSeek-V4.1-Flash-1017 at USD 0.10 per million input tokens. (deploy.com)

Claims the vendor makes about itself

Hive's rename post says its proprietary portfolio has over 20 models, which grew from content moderation into detecting IP-protected and AI-generated content. (thehive.ai, dated 2026-10-26) Hive says hundreds of organisations send billions of pieces of content a day through its APIs and that it operates its own datacenter infrastructure. (thehive.ai, dated 2026-10-26)

What our crawl saw

A GET for imagesift.com/.well-known/http-message-signatures-directory on 2026-11-05 returned HTTP 200 with an HTML page shell, not a key directory, so ImagesiftBot has no Web Bot Auth keys at that host. (imagesift.com)

Products and plans

ProductWhat it doesPrice modelFromAs of
ImageSift reverse image search Search for similar images across the index ImagesiftBot builds. No price recorded
Deploy inference platform Hosted open-source and proprietary models, including Hive's moderation and AI-content detection models. usage-based Per million input tokens for DeepSeek-V4.1-Flash-1017; Enterprise is custom. USD 0.10 2026-11-05

Standards

StandardStatusEvidence
RFC 9309 (robots.txt) Supports Directives addressed to ImagesiftBot are respected, Crawl-delay included; Googlebot rules apply when ImagesiftBot is not named. (imagesift.com)
Web Bot Auth No statement No key directory at imagesift.com (2026-11-05).
Published IP list No statement No IP list on the ImageSift page (2026-11-05).

Our checks

CheckResultMethodDate
Homepage fetch HTTP 200 GET with the crawl user agent after a robots.txt check 2026-11-05
Web Bot Auth key directory HTTP 200, text/html page shell, no keys GET https://imagesift.com/.well-known/http-message-signatures-directory with the crawl user agent after a robots.txt check 2026-11-05

What Hive does not do

  • ImageSift publishes no IP list or signature keys, so ImagesiftBot can be identified only by its user agent string (checked 2026-11-05).
  • ImageSift's page does not say whether images it collects are used to train Hive's or Deploy's models.

Reading notes

A site that allows Googlebot and blocks everything else through the * group still admits ImagesiftBot, because ImageSift applies its Googlebot rule when no group names its bot.

Sources

Last reviewed 2026-11-05