Free tool · runs in your browser
AI-ready badge for your site
Show visitors how your site treats AI: how many of 17 AI crawlers, from GPTBot and ClaudeBot to PerplexityBot and CCBot, your robots.txt lets in, and whether you serve an llms.txt. From our October 19, 2026 crawl of the top 10,000 sites, with the date on the badge. Copy one line of HTML or Markdown.
How it works
The badge is an SVG at /badge/<domain>.svg, made at build time from the same dataset as the AI crawler report, one file per site. It shows only what that crawl recorded: the robots.txt verdict for each crawler, parsed per RFC 9309, and the llms.txt check. A site whose robots.txt we could not read says so (“rules not read”) instead of guessing.
The embed links to your site's page in the report when it has one, else to the report, where visitors see the rule for each crawler. Making the badge here sends no request to the ContextFetch API.
Questions
What does the badge show?
What our crawl recorded for your domain: how many of the 17 AI crawlers we check your robots.txt allows everywhere, keeps out of parts of the site or blocks from the whole site; whether /llms.txt was a real llms.txt (a text file starting with a Markdown title); and the date of the crawl.
My site is not in the crawl. What does the badge say?
“Not checked yet”. The crawl covers the top 10,000 domains of the Majestic Million. The badge still loads (HTTP 200), so the embed never shows a broken image. The AI crawler checker reads any site's robots.txt live.
When does it change?
When we publish a new crawl. The date on the badge is the crawl it shows. Viewing the badge never fetches your site or calls the API.
Is allowing AI crawlers better than blocking them?
The badge does not judge: blocking AI crawlers is a valid choice, and the badge says what your robots.txt asks for. Green: every crawler allowed, or a real llms.txt. Amber: some crawlers kept out of all or part of the site. Gray: all blocked, rules not read, or no llms.txt found or checked.
What does the badge show?
What our crawl recorded for your domain: how many of the 17 AI crawlers we check your robots.txt allows everywhere, keeps out of parts of the site or blocks from the whole site; whether /llms.txt was a real llms.txt (a text file starting with a Markdown title); and the date of the crawl.
My site is not in the crawl. What does the badge say?
“Not checked yet”. The crawl covers the top 10,000 domains of the Majestic Million. The badge still loads (HTTP 200), so the embed never shows a broken image. The AI crawler checker reads any site's robots.txt live.
When does it change?
When we publish a new crawl. The date on the badge is the crawl it shows. Viewing the badge never fetches your site or calls the API.
Is allowing AI crawlers better than blocking them?
The badge does not judge: blocking AI crawlers is a valid choice, and the badge says what your robots.txt asks for. Green: every crawler allowed, or a real llms.txt. Amber: some crawlers kept out of all or part of the site. Gray: all blocked, rules not read, or no llms.txt found or checked.
More free tools
- URL to MarkdownPaste a URL, get the clean Markdown the API returns for it.GET /r/{url}
- Token counterA page's tokens as raw HTML, as Markdown and pruned to a question.GET /r/{url}
- AI crawler checkerWhich AI crawlers a site's robots.txt blocks: GPTBot, ClaudeBot and more.GET /r/{site}/robots.txt
- llms.txt generator and validatorCheck a site's llms.txt against the format, or draft one from its URLs.GET /r/{site}/llms.txt, POST /v1/map
- Site mapperA site's URLs from its links and sitemaps, without fetching the pages.POST /v1/map
- Search API cost calculatorMonthly cost at list price for ContextFetch, Exa and Tavily.
Give your agent the web
Your free key comes with 2,500 credits, no signup. One credit is one search or one page read.