Common Use Cases & Recipes

This guide shows you the most common web scraping scenarios and the exact parameters you need for each one. Use these ready-to-implement recipes to quickly solve your specific scraping challenges with the Universal Scraper API.

Use Case Overview

Use Case	Key Parameters	When You Need This
Load dynamic pages (SPAs, dashboards, infinite scroll)	js_render, wait_for, block_resources, js_instructions	When content loads via JavaScript or requires user interaction
Keep session/IP across requests	session_id, custom_headers	For multi-step workflows like account access → search → checkout
Submit forms or simulate user actions	js_instructions, js_render, wait_for	When you need to click buttons, fill forms, or navigate pages
Access geo-restricted content	premium_proxy, proxy_country	When content varies by location or is region-locked
Handle pagination and infinite scroll	js_render, js_instructions, wait_for	When content loads dynamically as you scroll or click “Load More”
Extract only specific data fields	css_extractor, autoparse, outputs	When you want structured data instead of full HTML
Capture API calls and background requests	json_response, js_render, wait_for	When the data you need comes from AJAX/XHR requests
Bypass anti-bot protection	premium_proxy, js_render, custom_headers, session_id	When sites use Cloudflare or other bot detection
Debug failed requests	original_status, allowed_status_codes, screenshot	When requests fail and you need to understand why
Optimize performance and costs	block_resources, outputs, wait_for	When you want faster responses and lower bandwidth usage

Code Recipes

1. Search for products using the search form

This example shows how to automatically fill and submit a search form, then wait for the search results to load.

Python

import requests

url = "https://api.zenrows.com/v1/"
params = {
    "apikey": "YOUR_ZENROWS_API_KEY",
    "url": "https://www.scrapingcourse.com/ecommerce/",
    "js_render": "true",
    "js_instructions": """[
        {"wait_for":"#wp-block-search__input-1"},
        {"fill":["#wp-block-search__input-1","hoodie"]},
        {"wait":500},
        {"click":"#block-2 > form > div > button"},
        {"wait_for":"#main"}
    ]""",
    "premium_proxy": "true",
}
response = requests.get(url, params=params)
print(response.text)

The js_instructions parameter fills the search field with “hoodie”, clicks the search button, and waits for the main content area to load with search results before returning the content.

Head to the JS Instructions Documentation for more details.

2. Maintain a Session Across Requests

Use the same session_id value to maintain cookies and session state across multiple requests.

Python

import requests

# Define session ID and API endpoint
session_id = "1234"
url = "https://api.zenrows.com/v1/"

# First request establishes a session
params1 = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/",
  "session_id": session_id
}
response1 = requests.get(url, params=params1)

# Second request reuses the same session
params2 = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/pagination",
  "session_id": session_id
}
response2 = requests.get(url, params=params2)

This approach is essential for workflows where you need to maintain the same IP across requests.

Learn more about the session_id parameter here.

3. Extract Specific Fields Only

Instead of getting the full HTML, extract only the data you need using CSS selectors.

Python

params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/ecommerce/",
  "css_extractor": """{"product_name":".product-name","price":".price"}""",
}

Returns clean, structured data:

JSON

{
  "price": [
    "$69.00",
    "$57.00",
    "$48.00",
    "$24.00",
    "$74.00",
    "$7.00",
    "$45.00",
    "$69.00",
    "$40.00",
    "$42.00",
    "$34.00",
    "$32.50",
    "$20.00",
    "$22.00",
    "$39.00",
    "$45.00"
  ],
  "product_name": [
    "Abominable Hoodie",
    "Adrienne Trek Jacket",
    "Aeon Capri",
    "Aero Daily Fitness Tee",
    "Aether Gym Pant",
    "Affirm Water Bottle",
    "Aim Analog Watch",
    "Ajax Full-Zip Sweatshirt",
    "Ana Running Short",
    "Angel Light Running Short",
    "Antonia Racer Tank",
    "Apollo Running Short",
    "Arcadio Gym Short",
    "Argus All-Weather Tank",
    "Ariel Roll Sleeve Sweatshirt",
    "Artemis Running Short"
  ]
}

Check the CSS Extractor Documentation for more details.

4. Scrape Localized Content

Access geo-restricted content or get localized pricing by routing your request through specific countries.

Python

params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://www.scrapingcourse.com/ecommerce/",
  "premium_proxy": "true",
  "proxy_country": "de"
}

This example routes your request through Germany, allowing you to see content as it appears to German users.

Find more details about geolocation on the Proxy Country Documentation.

5. Debugging a Blocked Request

When requests fail, use these parameters to understand what’s happening without losing the response data.

Check the actual HTTP status

Use the original_status parameter to see the real HTTP status code the target page returns.

Python

params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://example.com/blocked",
  "original_status": "true",
}
response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response)

Check more about the original_status parameter here.

Allow specific error status codes

Use the allowed_status_codes parameter to receive the actual HTML content even when the page returns error status codes.

Python

params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://example.com/blocked",
  "allowed_status_codes": "403,404,500",
}
response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)

Find out more on the Allowed Status Codes Documentation.

Add custom headers

Use custom headers to simulate browser behavior and avoid detection.

Python

params = {
  "apikey": "<YOUR_ZENROWS_API_KEY>",
  "url": "https://example.com/blocked",
}
headers = {
    'cookie': 'your-cookie',
    'referer': 'https://www.google.com',
}
response = requests.get('https://api.zenrows.com/v1/', params=params, headers=headers)
print(response.text)

ZenRows handles all browser-based headers. To find out more about it, check our Headers Documentation.

What’s Next

Ready to implement these recipes? Here are your next steps:

• Explore the Parameter Reference

• Follow the First Request Guide

• Jump back to Welcome