Integrate the REST API

REST routes use this base URL:

https://api.scrappycoco.ai/api/v1

Scrappycoco does not currently publish a language SDK. Application developers
call this REST API directly or generate a typed client from the public OpenAPI
document. The Scrappycoco CLI is an executable client, not an importable SDK.

Create a workspace API key in the Scrappycoco dashboard and keep it in a secret
manager. The asynchronous example also uses jq to capture the returned job ID.

export SCRAPPYCOCO_API_URL="https://api.scrappycoco.ai"
export SCRAPPYCOCO_API_KEY="scrappy_..."

1. List available capabilities

curl "$SCRAPPYCOCO_API_URL/api/v1/scrapers?available_only=true" \
  -H "X-API-Key: $SCRAPPYCOCO_API_KEY"

2. Inspect the selected capability

curl "$SCRAPPYCOCO_API_URL/api/v1/scrapers/web/extract_content" \
  -H "X-API-Key: $SCRAPPYCOCO_API_KEY"

Construct input from input_schema. Inspect each provider's
options_schema and place native settings under
provider_options.<provider_id>.

The example below extracts a web page. For other tasks, start with
Use cases and inspect the capability used by that guide.

3. Choose how to run

• Synchronous (POST /scrapers/execute): Get results in the same request.
  Use this when your workflow needs the data before continuing.
• Asynchronous (POST /scrapers/jobs): Submit a job and retrieve results
  later. Use this for longer-running work or when your client cannot keep the
  request open while the scraper runs.

Both options accept the same execution body and require Idempotency-Key.
The paths above are relative to the /api/v1 base URL.
The examples use fixed keys so an identical retry does not create another run.
Choose a new key when changing the request or intentionally running it again.

Get results in the same request

curl --fail-with-body -sS -X POST \
  "$SCRAPPYCOCO_API_URL/api/v1/scrapers/execute" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $SCRAPPYCOCO_API_KEY" \
  -H "Idempotency-Key: extract-sync-example-v1" \
  -d '{
    "source": "web",
    "capability": "extract_content",
    "input": {"url": "https://example.com"},
    "providers": ["firecrawl_scrape"],
    "provider_options": {
      "firecrawl_scrape": {
        "formats": ["markdown"],
        "only_main_content": true
      }
    },
    "limit": 10
  }'

The response contains records directly, alongside status, provider
attempts, usage, and next_action. No job polling is needed. Check status
and attempts for partial results or provider failures; the caller still
decides whether the records satisfy the request.

For content formats and provider-specific output limits, see
Extract web content.

Allow enough time for the scraper to finish within your client's request
timeout, or use the asynchronous option below. See the
synchronous endpoint reference
for the full request and response schemas.

Queue a run and retrieve results later

JOB_RESPONSE="$(curl --fail-with-body -sS -X POST \
  "$SCRAPPYCOCO_API_URL/api/v1/scrapers/jobs" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $SCRAPPYCOCO_API_KEY" \
  -H "Idempotency-Key: extract-example-v1" \
  -d '{
    "source": "web",
    "capability": "extract_content",
    "input": {"url": "https://example.com"},
    "providers": ["firecrawl_scrape"],
    "provider_options": {
      "firecrawl_scrape": {
        "formats": ["markdown"],
        "only_main_content": true
      }
    },
    "limit": 10
  }')"

JOB_ID="$(printf '%s' "$JOB_RESPONSE" | jq -r '.job_id')"
printf 'Queued job: %s\n' "$JOB_ID"

The API returns HTTP 202 and a job_id. The example captures the response
and extracts JOB_ID with jq before the next command uses it.

The example requests main content only. If the caller needs the original HTML
<head> for titles, meta tags, canonical URLs, robots directives, or JSON-LD,
use "formats":["rawHtml"] with "only_main_content":false.

Read the queued job

curl --fail-with-body -sS "$SCRAPPYCOCO_API_URL/api/v1/jobs/$JOB_ID" \
  -H "X-API-Key: $SCRAPPYCOCO_API_KEY"

Poll with backoff until status is completed, failed, or cancelled. A
completed job includes normalized records, provider attempts, exact effective
options, billing state, and next_action.

Idempotency

Billable execution POSTs require Idempotency-Key. Generate a unique value for
each logical request and reuse it only for an identical transport retry. A key
reused with different input returns 409.

See the generated API Reference for every endpoint and
schema.