Extract web content

Use web.extract_content to read a page through a provider you select.
Start with readable content for summaries and research. Request full HTML when
you need metadata such as canonical links, robots directives, or JSON-LD.

Before you start

Set up the CLI and sign in. These examples use Firecrawl;
check that it is available for your request:

scrappycoco catalog inspect web.extract_content --json

Ask your connected AI agent

Use Scrappycoco to extract the main content from https://example.com.
Return the readable text, the provider used, and any missing or truncated content.

Extract a page with the CLI

scrappycoco run web.extract_content \
  --input '{"url":"https://example.com"}' \
  --provider firecrawl_scrape \
  --provider-options '{"firecrawl_scrape":{"formats":["markdown"],"only_main_content":true}}' \
  --limit 10 \
  --output result.json \
  --json

only_main_content: true may remove the entire HTML <head>. For SEO,
structured-data, or indexing checks that need titles, meta tags, canonicals,
robots directives, or JSON-LD, request rawHtml and set it to false:

scrappycoco run web.extract_content \
  --input '{"url":"https://example.com"}' \
  --provider firecrawl_scrape \
  --provider-options '{"firecrawl_scrape":{"formats":["rawHtml"],"only_main_content":false}}' \
  --output full-page.json --json

Read your result

Open result.json (or full-page.json for the HTML example). Inspect records
and each record's outputs and primary_format for the requested content.
Check attempts for provider failures and any truncation metadata before using
the result as a complete copy of the page.

With Browserbase, content records include outputs.text and outputs.html.
Text is capped at 50,000 characters and HTML at 2,000,000. Inspect
text_truncated and html_truncated in the native attempt response.

Use another interface

The REST guide includes the equivalent extraction request and
explains synchronous responses and queued jobs. In MCP, inspect
web.extract_content with get_scraper and pass the same input and provider
options to run_scraper.

To control cached content, see Data freshness.
To test a different provider or rendering option, see
Compare and save configurations.