---
title: "omgili: user agent, token, block rate | Agent Ready Crawlers"
description: "omgili from Webz.io: exact user-agent string, robots.txt token omgili, other purpose and how to verify it. 15.7% of top sites block it. Read 2026-11-04."
canonical: https://agentreadycrawlers.com/crawlers/omgili
site: "Agent Ready Crawlers"
last_updated: 2026-11-04
section: "Crawlers"
---

# omgili: user agent, token, block rate | Agent Ready Crawlers

> omgili from Webz.io: exact user-agent string, robots.txt token omgili, other purpose and how to verify it. 15.7% of top sites block it. Read 2026-11-04.

Source: https://agentreadycrawlers.com/crawlers/omgili
Site: Agent Ready Crawlers
Last reviewed: 2026-11-04

Bot from Webz.io

# omgili (Omgilibot)

In short

omgili (Omgilibot) is Webz.io's bot, filed under Other. The token to write in robots.txt is omgili. The operator publishes nothing a site can check. 15.7 percent of 5,938 top-10,000 sites with a parsed robots.txt disallow it at the root (crawl of 2026-11-04). Webz.io says its newer crawlers adhere to robots.txt exclusions (2024-07-17 post introducing the "Webzio Duo").

Operator

Webz.io Vendor profile: [Webz.io](https://agentreadycrawlers.com/vendors/webz-io/)

Purpose

Other [All other rows](https://agentreadycrawlers.com/crawlers/purpose/other/). Link previews, ad checks, dataset crawls, SEO tools and legacy tokens that fit none of the five purposes.

User-agent string

`omgili/0.5 +https://omgili.com`

robots.txt token

`omgili` Matched case-insensitively (RFC 9309). The token and the user-agent string are different things.

Respects robots.txt

Yes, per operator

Webz.io says its newer crawlers adhere to robots.txt exclusions (2024-07-17 post introducing the "Webzio Duo"). [Source](https://webz.io/bot.html), read 2026-11-04.

Verification

Nothing published

The operator publishes no list, reverse DNS pattern or key directory a site owner can check. [Documentation](https://webz.io/blog/web-data/what-is-the-omgili-bot-and-why-is-it-crawling-your-website/), read 2026-11-04.

IP ranges

Not published.

Rate behaviour

Not published.

Block rate

**15.7%** of 5,938 top-10,000 sites with a parsed robots.txt disallow `omgili` at the root (933 sites). 11.8% name it; 3.9% refuse it only through a \* group. Rank 8 of 31 tokens measured.

Across all 8,624 parsed robots.txt files in the crawl: 11.8% (9.0% named). Crawl of 2026-11-04; [report](https://agentreadycrawlers.com/block-rates/).

Documentation

[webz.io](https://webz.io/blog/web-data/what-is-the-omgili-bot-and-why-is-it-crawling-your-website/), read 2026-11-04.

Note. Web data vendor whose feeds are sold for AI uses. The 2017 post gives the omgili/0.5 string.

## Allow and refuse lines

### To allow omgili

User-agent: omgili
Allow: /

### To refuse omgili

User-agent: omgili
Disallow: /

Refusing or allowing in robots.txt states a preference. Enforcement of a refusal belongs in your WAF; see how to verify the identity above.

## Webz.io in the Verification Ledger

Publishes nothing for verification. [All operators](https://agentreadycrawlers.com/verification/).

IP lists

publishes nothing

Reverse DNS

publishes nothing

Key directory

publishes nothing

Cloudflare

not checked

Visa TAP registry

no public operator-level registration found (checked 2026-11-04)

Mastercard Agent Pay

no public operator-level registration found (checked 2026-11-04)

## Sources

-   [webz.io: what is the omgili bot and why is it crawling your website](https://webz.io/blog/web-data/what-is-the-omgili-bot-and-why-is-it-crawling-your-website/) read 2026-11-04
-   [webz.io: bot](https://webz.io/bot.html) read 2026-11-04

Last reviewed 2026-11-04