Skip to main content
The Batch Scraper API is in public beta, available to every ZenRows account. You need an active ZenRows API key. Replace YOUR_ZENROWS_API_KEY in every example with your key.

​
Basics

  • Base URL: https://async.api.zenrows.com/v1
  • Authentication: send X-API-Key: YOUR_ZENROWS_API_KEY on every request. The key travels in a header, never in the URL.
  • Bodies: send JSON with Content-Type: application/json and receive JSON. Errors come back as application/problem+json, a standard JSON error format (RFC 7807) with a stable code you can branch on.
The Node.js examples below reuse this small wrapper, which adds authentication, sets the Content-Type header when there is a body, and turns error responses into thrown objects:
Node.js

​
1. Create your jobs

There are several ways to create a job. Pick the one that matches how your URLs arrive.

​
Submit a job with all URLs up front

Create a closed job (one where you know all URLs up front) with POST /jobs. The response includes a job_id and the initial run status.
POST /jobs returns 201 Created for submissions under 10,000 inline tasks and 202 Accepted for 10,000 or more tasks (or a CSV input). The request shape and response body are identical, so treat both the same: store job_id, then poll until stats.completed >= stats.total (or use a webhook). On a 202, stats.total is correct immediately, but GET /jobs/{id}/results may return partial pages for a few seconds while the task rows finish writing.

​
Configure the scrape and per-URL options

zenrows_params applies to every URL in the job. A per-task zenrows_params overrides the job-level keys for that URL, so you can set the common case once and tune exceptions such as a different proxy_country. Common keys:
The API returns your external_id unchanged on every result, so you can match results back to your own records without tracking the API’s task IDs. It does not need to be unique.

​
Open jobs: adding tasks in batches

Open (queue-mode) jobs are still being finalized during the beta and may not be available yet. If POST /jobs with status: "open" returns a 503, use a closed job or a CSV upload for now.
Create an open job, add tasks with POST /jobs/{job_id}/tasks over time, then close it by sending last_batch: true on the final batch. Each add-tasks call accepts up to 10,000 URLs.
last_batch: true (or POST /jobs/{id}/close) locks the job, and later adds return 409. To abandon a run that is still in flight, call POST /jobs/{id}/stop (see Control the current run). A very large submission also keeps ingesting task rows for a few seconds after it responds; POST /jobs/{id}/tasks can return 409 during that window, so retry shortly.

​
Large inputs: CSV upload

For lists too large to inline, upload a CSV in three steps:
  1. Create an input slot with POST /job_inputs.
  2. PUT the file to the returned presigned URL using the exact headers it specifies
  3. Submit a job referencing file_input_id.
The fields map points the canonical url (required) and external_id (optional) fields at your CSV columns by header name (with header: true) or 0-based index.
Limits: 50 MB / 100,000 rows. Upload slots live 24 hours.

​
Scheduled jobs

A scheduled job stores its URLs as a template, and each fire creates a new run. Provide exactly one of at, rate, or calendar (plus a timezone for at and calendar). Times of day must be full hours. The request below schedules a job to run every 6 hours.
For a calendar schedule, replace the schedule object. The example below runs at 09:00 and 18:00 on Mon/Wed/Fri in Berlin time. Use an at object instead for a one-shot run at a single wall-clock time.
Manage a schedule with PUT /jobs/{id}/schedule to replace it, and POST /jobs/{id}/schedule/state with {"schedule_state": "paused"} or {"schedule_state": "active"} to pause or resume.

​
2. Track your jobs

Check progress and find your jobs once they are running.

​
Check job status

Read a job with GET /jobs/{job_id}. Progress lives in latest_run.stats, where completed equals successful plus failed.
To wait for a run to finish, poll this endpoint until latest_run.status reaches a terminal state (completed, stopped, or deleted), backing off between calls rather than running a tight loop.

​
List and paginate jobs

List your jobs with GET /jobs, optionally filtered by status and type, and page with cursor until next_cursor is null.
The cursor loop is the same as for results, and GET /jobs/{id}/runs pages identically.

​
Control the current run

Stop, pause, or resume a job’s latest run. Each endpoint is a POST with no request body and returns the refreshed run object on 200.
Stop and cancel. /stop moves the latest run from running or pending to the terminal state stopped. No new tasks are picked up, tasks already in flight may still finish and record a result, and pending tasks are left as-is (not re-queued, not failed). Result bodies are kept; to free storage, follow up with DELETE /jobs/{id}/runs/{run_id}. It is idempotent on an already-stopped run, returns 409 if the run is completed or deleted (use a rerun to start fresh), and 404 on a deleted job. /cancel is a pure alias for /stop. Pause and resume. /pause reversibly suspends the latest run: the dispatcher stops polling its queue, while the run’s status (running or pending) is unchanged, since pause is orthogonal to status. Tasks already in flight may still settle. /resume restarts polling. Both are idempotent (pausing a paused run, or resuming an active run, is a no-op), and both return 409 when the latest run is terminal.
In-flight gateway calls still consume usage while a pause takes effect. For a hard cost-stop, use /stop (or /cancel) instead.
This is not the same as pausing a schedule. For scheduled jobs, POST /jobs/{id}/schedule/state controls whether new runs fire. The /pause and /resume endpoints here act on the current run.

​
3. Collect your results

Retrieve the scraped content once a run completes.

​
Download content

List a run’s results with GET /jobs/{job_id}/results, filtered by status (successful, failed, or all) and paged with the cursor query parameter until next_cursor is null.
Each row’s result_url is a presigned link valid for 2 hours, so fetch it promptly.
To page through every result and download each body, loop over the pages until next_cursor is null, fetching each presigned result_url promptly.
You can also fetch one task’s body directly with GET /jobs/{job_id}/tasks/{task_id}/content.
  • 200: returns the content
  • 409: means the task is still processing
  • 422: means the task failed and the body holds the stored error.
A failed task’s results row already carries row.error ({ code, detail, ... }), so you rarely need the content endpoint to inspect a failure. During the beta, treat the error code as a hint: it does not yet reliably distinguish blocked, not-found, and timeout.

​
Bulk export (ZIP)

Bundle an entire run’s result bodies into a single ZIP. Start the export with POST /jobs/{job_id}/runs/{run_id}/exports, poll the export until status is completed, then download the presigned download_url.
Exports expire 12 hours after creation, the API presigns download_url fresh on each poll, and a run whose combined results exceed 1 GB fails with a clear error instead of producing a partial archive.
Start the export, poll until it is completed, then download the ZIP.

​
4. Webhooks and notifications

Get notified when a run finishes, and secure those callbacks.

​
Set up webhooks

Attach a webhook so the API POSTs a run.completed event to your HTTPS URL when a run finishes. Set signature: true for HMAC-signed deliveries (see Manage HMAC keys). You can attach it on the submit body, replace it later with PUT /jobs/{id}/webhook, remove it with DELETE /jobs/{id}/webhook, and probe a receiver with POST /webhook/test.
The API delivers each event at least once, so deduplicate on the X-ZenRows-Event-Id header. Webhooks are a convenience; the run state and result URLs remain the reliable source of truth.

​
Manage HMAC keys

Signed webhook deliveries need an active HMAC key for your organization. Rotation uses two slots, an active key plus a candidate, so you can roll keys without downtime.
  • POST /hmac/keys/rotate creates the first key or stages a candidate (the secret is returned once)
  • POST /hmac/keys/rotate/finalize promotes the candidate and retires the old key
  • DELETE /hmac/keys/rotate discards a staged candidate
  • GET /hmac/keys lists key metadata without secrets

​
Verify signed webhooks

When signature: true, each delivery includes an X-Signature header in the form t=<unix>,v1=<hex>,kid=<active>, where t is the timestamp, v1 is the signature, and kid identifies the key used. To verify a delivery:
  1. Look up the base64 secret that matches the delivery’s kid.
  2. Compute HMAC-SHA256(secret, "<t>.<raw_body>"), signing the timestamp, a literal dot, and the raw request body.
  3. Compare the result to v1. If they match, the delivery is authentic. Always sign the raw request body, never a re-serialized version, or the signature will not match. If signature is false, no X-Signature header is sent, so authenticate the callback another way, such as a secret path in the URL. Either way, deduplicate on the X-ZenRows-Event-Id header, since the same delivery can arrive more than once.
The examples below compute the expected signature so you can compare it to v1 in your receiver.

​
5. Reruns and retrying failures

POST /jobs/{job_id}/rerun replays a job as a new run. Add ?status=failed to re-run only the failures (append ,pending to also include tasks that never started). Already-successful tasks carry over, so you only re-scrape, and only pay for, what failed. The response reports retried_tasks and inherited_tasks.

​
6. Cost

Estimate spend before you submit, and read actual spend after a run.

​
Estimate cost

Pricing is per successful request by tier, and the rate card is small and stable, so you can estimate cost on the client with no API call.
  • Basic request (base)
  • 5x multiplier (JavaScript rendering)
  • 10x multiplier (premium proxy)
  • 25x multiplier (both)
  • a 1x to 25x range for mode=auto
Sum the tier across your tasks. The estimate assumes each URL succeeds once, so it is a single number unless a task uses mode=auto.
The estimate assumes each URL succeeds once, so it is a single number unless a task uses mode=auto. For more details on multipliers, see the Pricing Page.

​
Check actual cost

Once a run finishes, you can read the usage it actually consumed at two levels:
  • Run total: the spend object on latest_run.stats, read from the GET /jobs/{id} response shown below.
  • Per task: the spend object on each result row, where spend.total.credits is the cost across all attempts and spend.last_attempt.credits is the cost of the most recent attempt. Read these while paging results, as shown in Download content.
Only successful (status 200, 404 and 410) requests cost usage, so a run with failures costs less than its estimate. Treat this figure as an indicative signal. Your ZenRows account statement remains the authoritative record for billing.

​
7. Reference

Behaviors and error details that apply across the API.

​
Idempotency

Send an Idempotency-Key header on a submit to guard against accidentally creating duplicate jobs. Each key can be used once: the first request with a given key creates the job, and any later request that reuses the same key returns 409 idempotency_key_conflict, whether or not the body is identical. The API does not store or replay the original response.
An idempotency key is not a safe-retry token, because reusing it always returns 409. If a submit fails midway, such as a dropped connection or a 503, do not resend with the same key. First check whether the job was created by listing your recent jobs, then either use its job_id or submit again with a new key. A late 503 can leave a visible job with a stopped run, which is safe to DELETE.

​
Error handling reference

Errors use RFC 7807 problem JSON with a stable code you can branch on. Task-level failures are not HTTP errors; they appear as failed rows in the results, each with its own error object. A successful POST /jobs returns 201 Created (synchronous) or 202 Accepted (asynchronous, for 10,000+ tasks or a CSV input); both carry the same response body. On a non-2xx response, read the code and, for a 400, the invalid_tasks array from the problem body.

​
Troubleshooting

Common issues you may hit while integrating. See the full Troubleshooting page for the complete list.
  • 401 unauthenticated: send your key in the X-API-Key header (never in the URL); a 401 means the key is missing or invalid.
  • 400 invalid_argument: the body failed validation. Read the invalid_tasks array to see which URL failed and why.
  • 429 quota_exceeded: you have 3 jobs running already. Wait for one to finish, since 3 concurrent jobs is the limit.
  • 503 when opening a job: open (queue-mode) jobs are still being finalized during the beta. Use a closed job or a CSV upload for now.
  • Download link returns an error: each result_url is valid for 2 hours. Re-list the results to get a fresh link rather than reusing a stored one.
  • Webhook never arrives: use an HTTPS receiver, test it with POST /webhook/test, and deduplicate on the X-ZenRows-Event-Id header, since deliveries are at-least-once.

​
FAQ (Frequently Asked Questions)

Quick answers for developers. See the full FAQ for more.
All requests go to https://async.api.zenrows.com/v1, authenticated with the X-API-Key header. The Batch Scraper API uses the same API key as the Universal Scraper API.
Any language with an HTTP client. The examples in this guide cover Python, Node.js, Java, PHP, Go, Ruby, and cURL, and the same REST calls work anywhere.
Up to 100,000 URLs per job submit, 10,000 per add-tasks call on an open job, and 3 active jobs concurrently. CSV uploads support 50 MB / 100,000 rows.
No. Each key can be used once, and reusing it always returns 409 idempotency_key_conflict. If a submit fails midway, check whether the job was created before resending with a new key.
In usage, per successful request, at the same rates as the Universal Scraper API. Failed requests are never charged.