Skip to main content
Point your assistant at a directory-style site, a listings page, a member roster, a catalogue, and have it walk every entry.

​
Setup

Connect the MCP server first, see setup.

​
What you’ll do

  • Crawl a site from a seed URL
  • Read the returned pages as clean markdown rather than raw HTML
  • Have the assistant pull out the fields you care about from what it read

​
Example exchange

“Go through this directory site and pull out every listing’s name, location and contact page.” The assistant calls outcrawl_crawl with the directory’s seed URL. outcrawl_crawl folds the whole stream into one response, so the assistant gets every page’s content back from that single call rather than polling a separate results tool, there is no outcrawl_crawl_results tool, only the REST API and CLI expose the id-based lifecycle. It then reads the fields you asked for out of the returned pages.

​
What it can’t do

Each page crawled is 1 credit, the same as a direct scrape. There is no separate “extraction” capability with its own price, this is plain crawling plus reading the structured content already returned.