Free tool · no key · runs on the live API
Site mapper: list a website's URLs
Enter a site to list its URLs from the homepage's links and the site's sitemaps, without fetching the pages. See how they split into sections, then copy the list or download it as a text file.
How it works
The tool sends one request to POST https://api.contextfetch.dev/v1/map with the start URL, a limit and an optional path filter. The response lists the URLs and says how many came from the start page's links and how many from sitemaps. The sections are the URLs grouped by the first segment of their path, counted in your browser.
Agents can make the same call to see what a site has before reading any of it. The ContextFetch MCP server has it as the contextfetch_map tool, and the playground shows the code in curl, Python and JavaScript.
Questions
Where do the URLs come from?
The start page's links to the same site, then the sitemaps its robots.txt names (or /sitemap.xml when it names none). Sitemap index files are followed, up to 10 sitemap files per map, and gzipped sitemaps are read. www. and the bare domain count as the same site.
Does it fetch every page?
No. It reads only the start page, robots.txt and up to 10 sitemap files. To get the pages themselves as Markdown, use the crawl endpoint or the reader.
How many URLs can it list?
Up to 5,000 per map, and a map is one request whatever the limit: one of your 25 free keyless requests a day, or one credit with a key. A map stops after 25 seconds and returns what it has found by then.
What does the path filter do?
It keeps only URLs whose path matches the regular expression, such as ^/docs/ for the documentation. The API also takes several include and exclude patterns.
Where do the URLs come from?
The start page's links to the same site, then the sitemaps its robots.txt names (or /sitemap.xml when it names none). Sitemap index files are followed, up to 10 sitemap files per map, and gzipped sitemaps are read. www. and the bare domain count as the same site.
Does it fetch every page?
No. It reads only the start page, robots.txt and up to 10 sitemap files. To get the pages themselves as Markdown, use the crawl endpoint or the reader.
How many URLs can it list?
Up to 5,000 per map, and a map is one request whatever the limit: one of your 25 free keyless requests a day, or one credit with a key. A map stops after 25 seconds and returns what it has found by then.
What does the path filter do?
It keeps only URLs whose path matches the regular expression, such as ^/docs/ for the documentation. The API also takes several include and exclude patterns.
More free tools
- URL to MarkdownPaste a URL, get the clean Markdown the API returns for it.GET /r/{url}
- Token counterA page's tokens as raw HTML, as Markdown and pruned to a question.GET /r/{url}
- AI crawler checkerWhich AI crawlers a site's robots.txt blocks: GPTBot, ClaudeBot and more.GET /r/{site}/robots.txt
- AI-ready badgeA badge for your site with its AI crawler rules and llms.txt status from our crawl.
- llms.txt generator and validatorCheck a site's llms.txt against the format, or draft one from its URLs.GET /r/{site}/llms.txt, POST /v1/map
- Search API cost calculatorMonthly cost at list price for ContextFetch, Exa and Tavily.
Give your agent the web
Your free key comes with 2,500 credits, no signup. One credit is one search or one page read.