Make Your First Request with ZenRows' Universal Scraper API

Follow this step-by-step guide to make your first web scraping API call with ZenRows in Python, Node.js, Go, Java, PHP, Ruby, or cURL.

Learn how to extract data from any website using ZenRows’ Universal Scraper API. This guide walks you through creating your first scraping request that can handle sites at any scale.

ZenRows’ Universal Scraper API is designed to simplify web scraping. Whether you’re dealing with static content or dynamic JavaScript-heavy sites, you can get started in minutes with any programming language that supports HTTP requests.

1. Set Up Your Project

Set Up Your Development Environment

Before diving in, ensure you have the proper development environment and required HTTP client libraries for your preferred programming language. ZenRows works with any language that can make HTTP requests.

While previous versions may work, we recommend using the latest stable versions for optimal performance and security.

Python

Python 3 is recommended, preferably the latest version. Consider using an IDE like PyCharm or Visual Studio Code with the Python extension.

# Install Python (if not already installed)
# Visit https://www.python.org/downloads/ or use package managers:

# macOS (using Homebrew)
brew install python

# Ubuntu/Debian
sudo apt update && sudo apt install python3 python3-pip

# Windows (using Chocolatey)
choco install python

# Install the requests library
pip install requests

If you need help setting up your environment, check out our detailed Python web scraping setup guide.

Node.js

Node.js 18 or higher is recommended, preferably the latest LTS version. Consider using an IDE like Visual Studio Code or IntelliJ IDEA to enhance your coding experience.

# Install Node.js (if not already installed)
# Visit https://nodejs.org/ or use package managers:

# macOS (using Homebrew)
brew install node

# Ubuntu/Debian (using NodeSource)
curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
sudo apt-get install -y nodejs

# Windows (using Chocolatey)
choco install nodejs

# Install the axios library
npm install axios

If you need help setting up your environment, check out our detailed Node.js scraping guide

Java

Java 8 or higher is recommended, preferably the latest LTS version, and a build tool like Maven or Gradle. IDEs like IntelliJ IDEA or Eclipse provide excellent Java development support.

# Install Java (if not already installed)
# Visit https://adoptium.net/ or use package managers:

# macOS (using Homebrew)
brew install openjdk

# Ubuntu/Debian
sudo apt update && sudo apt install openjdk-17-jdk

# Windows (using Chocolatey)
choco install openjdk

For Maven projects, add this dependency to your pom.xml:

xml

<dependency>
    <groupId>org.apache.httpcomponents.client5</groupId>
    <artifactId>httpclient5-fluent</artifactId>
    <version>5.2.1</version>
</dependency>

PHP

PHP 7.4 or higher with cURL extension enabled is recommended, preferably the latest stable version. Consider editors like PhpStorm or Visual Studio Code with PHP extensions.

# Install PHP (if not already installed)
# Visit https://www.php.net/downloads or use package managers:

# macOS (using Homebrew)
brew install php

# Ubuntu/Debian
sudo apt update && sudo apt install php php-curl

# Windows (using Chocolatey)
choco install php

PHP comes with cURL built-in, so no additional packages are needed for basic HTTP requests.

Go

Go 1.16 or higher is recommended, preferably the latest stable version. Visual Studio Code with the Go extension or GoLand provide excellent Go development environments.

# Install Go (if not already installed)
# Visit https://golang.org/dl/ or use package managers:

# macOS (using Homebrew)
brew install go

# Ubuntu/Debian
sudo apt update && sudo apt install golang-go

# Windows (using Chocolatey)
choco install golang

Go’s standard library includes the net/http package, so no additional dependencies are needed for HTTP requests.

Ruby

Ruby 2.7 or higher is recommended, preferably the latest stable version. Consider editors like RubyMine or Visual Studio Code with Ruby extensions.

# Install Ruby (if not already installed)
# Visit https://www.ruby-lang.org/en/downloads/ or use package managers:

# macOS (using Homebrew)
brew install ruby

# Ubuntu/Debian
sudo apt update && sudo apt install ruby-full

# Windows (using Chocolatey)
choco install ruby

# Install the faraday gem
gem install faraday

cURL

Typically pre-installed on most systems. If not available, install using your system’s package manager.

# cURL is typically pre-installed on most systems. If not, install it:

# macOS (using Homebrew)
brew install curl

# Ubuntu/Debian
sudo apt update && sudo apt install curl

# Windows (using Chocolatey)
choco install curl

No additional packages needed - cURL works directly from the command line.

Get Your API Key

Sign up for a free ZenRows account and get your API key from the Playground dashboard. You’ll need this key to authenticate your requests.

2. Make Your First Request

Start with a simple request to understand how ZenRows works. We’ll use the HTTPBin.io/get endpoint to demonstrate how ZenRows processes requests and returns data.

Python

# pip install requests
import requests

url = 'https://httpbin.io/get'
apikey = 'YOUR_ZENROWS_API_KEY'
params = {
    'url': url,
    'apikey': apikey,
    'mode': 'auto',
}

response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)

Replace YOUR_ZENROWS_API_KEY with your actual API key and run the script:

Python

python your_script.py

Expected Output

The script will print the contents of the website, for HTTPBin.io/get it’s something similar to this:

Response

{
    "args": {},
    "headers": {
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/137.0.0.0 Safari/537.36",
        // additional headers omitted for brevity...
    },
    "origin": "38.154.5.224:6693",
    "url": "http://httpbin.io/get"
}

Perfect! You’ve just made your first web scraping request with ZenRows.

3. Scrape More Complex Websites

Modern websites often use JavaScript to load content dynamically and employ sophisticated anti-bot protection. ZenRows provides powerful features to handle these challenges automatically.

Use Adaptive Stealth Mode

The simplest way to scrape complex or protected sites in code is to use Adaptive Stealth Mode (mode=auto). ZenRows analyzes each request and automatically applies the right combination of JavaScript rendering and Premium Proxies only when needed, so you get reliable results without having to tune parameters yourself. Add mode=auto to your request:

Python

import requests

url = 'https://www.scrapingcourse.com/antibot-challenge'
apikey = 'YOUR_ZENROWS_API_KEY'

params = {
    'url': url,
    'apikey': apikey,
    'mode': 'auto',
}

response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)

Optional parameters with Adaptive Stealth Mode: When you use mode=auto, only the managed parameters (js_render and premium_proxy) are disabled to prevent conflicts. You can still use other API parameters alongside Adaptive Stealth Mode, such as:

• Proxy country (proxy_country): For region-restricted or geo-specific content, set the country of the IP (e.g., proxy_country=us). When a proxy country is set, ZenRows automatically applies Premium Proxies to test the most reliable configuration. If our system detects that another proxy type or country delivers better results for your request, we may switch it automatically to maximize success rates.
• JavaScript rendering wait (wait or wait_for): For slow-loading or dynamic content, add wait (milliseconds after page load) or wait_for (CSS selector). When a wait time is set, ZenRows automatically applies JavaScript rendering on the configuration.
• Other parameters: css_extractor, session_id, custom headers, and other API parameters remain available for use with mode=auto.

Example using optional parameters with Adaptive Stealth Mode:

Python

params = {
    'url': url,
    'apikey': apikey,
    'mode': 'auto',
    'proxy_country': 'us',  # Optional: specify proxy location
    'wait': '3000',         # Optional: wait 3 seconds when JS rendering is used
}

Use the Request Playground (easy mode)

The Request Playground in your ZenRows dashboard offers an easy mode that uses Adaptive Stealth Mode by default. Enter your target URL and run a request. There’s no need to choose JavaScript rendering or Premium Proxies yourself. ZenRows picks the proper configuration for each URL.

Benefits:

• Test any URL visually and see the result immediately
• Get a ready-to-use cURL command or code snippet with mode=auto included
• Reliable results on complex or protected sites without manual parameter tuning

Enter the target URL (for this demonstration, the Anti-bot Challenge page) in the URL to Scrape field and use easy mode to get started.

Use the Request Playground (custom mode)

For fine-grained control when you know exactly which features a site needs, use the Request Playground’s custom mode to set parameters yourself. You can manually configure JavaScript rendering, Premium Proxies, and other options, or select Adaptive Stealth Mode (mode=auto) to let ZenRows choose automatically.

Use Premium Proxies

Premium Proxies provide access to over 55 million residential IP addresses from 190+ countries with 99.9% uptime, ensuring the ability to bypass sophisticated anti-bot protection.

Enable JavaScript Rendering

JavaScript Rendering uses a real browser to execute JavaScript and capture the fully rendered page. This is essential for modern web applications, single-page applications (SPAs), and sites that load content dynamically.

Combine Features for Maximum Success

For the most protected sites, enable both JavaScript Rendering and Premium Proxies. This provides the highest success rate for challenging targets.

Python

import requests

url = 'https://www.scrapingcourse.com/antibot-challenge'
apikey = 'YOUR_ZENROWS_API_KEY'

params = {
    'url': url,
    'apikey': apikey,
    'js_render': 'true',
    'premium_proxy': 'true',
}

response = requests.get('https://api.zenrows.com/v1/', params=params)
print(response.text)

This code sends a GET request to the ZenRows API endpoint with your target URL and authentication. The js_render parameter enables JavaScript processing, while premium_proxy routes your request through residential IP addresses.

Use-case recipes

Here are a few quick recipes you can adapt:

• Form submission → Use js_render + js_instructions.
• Keep session across requests → Add session_id to maintain session state and IP consistency.
• Extract structured fields only → Use css_extractor to return just the fields you need.

See the full Common Use Cases & Recipes guide.

Run Your Application

Execute your script to test the scraping functionality and verify that your setup works correctly.

Python

python your_script.py

Example Output

Run the script, and ZenRows will handle the heavy lifting by rendering the page’s JavaScript and routing your request through premium residential proxies. The response will contain the entire HTML content of the page:

HTML

<html lang="en"><head>
    <title>Antibot Challenge - ScrapingCourse.com</title>
</head><body>
    <h2>You bypassed the Antibot challenge! :D</h2>
</body></html>

Congratulations! You now have a ZenRows integration that can scrape websites at any scale while bypassing anti-bot protection. You’re ready to tackle more advanced scenarios and customize the API to fit your scraping needs.

Troubleshooting

Request failures can happen for various reasons. While some issues can be resolved by adjusting ZenRows parameters, others are beyond your control, such as the target server being temporarily down.

Below are some quick troubleshooting steps you can take:

1. Check the Error Code and Error Message

   When faced with an error, it’s essential first to check the error code and message for indications of the error. The most common error codes are:

   • 401 Unauthorized
     Your API key is missing, incorrect, or improperly formatted. Double-check that you are sending the correct API key in your request headers.

   • 429 Too Many Requests
     You have exceeded your concurrency limit. Wait for ongoing requests to finish before sending new ones, or consider upgrading your plan for higher limits.

   • 413 Content Too Large
     The response size exceeds your plan’s limit. Use CSS selectors to extract only the needed data, reducing the response size.

   • 422 Unprocessable Entity
     Your request contains invalid parameter values, or anti-bot protection is blocking access. Review the API documentation to ensure all parameters are correct and supported.

   Get more information on the API Error Codes page.
2. Check if the Site is Publicly Accessible

   Some websites may require a session, so verifying if the site can be accessed without logging in is a good idea. Open the target page in an incognito browser to check this.

   You must handle session management in your requests if authentication credentials are required. You can learn how to scrape a website that requires authentication in our guide: Web scraping with Python.

3. Verify the Site is Accessible in Your Country

   Sometimes, the target site may be region-restricted and only accessible to specific locations. ZenRows automatically selects the best proxy, but if the site is only available in concrete regions, specify a geolocation using proxy_country.

   Here’s how to choose a proxy in the US:

   Python

   params = {
       'premium_proxy': 'true',
       'proxy_country': 'us' # <- choose a premium proxy in the US
       # other configs...
   }
   response = requests.get('https://api.zenrows.com/v1/', params=params)
   
   

   If the target site requires access from a specific region, adding the proxy_country parameter will help.

   Check out more about it on our Geolocation Documentation Page.
4. Add Pauses to Your Request

   You can also enhance your request by adding options like wait or wait_for to ensure the page fully loads before extracting data, improving accuracy.

   Python

   # Use a wait parameter or a wait_for parameter
   params = {
       'wait': '3000', # <- Adds a delay of 3 seconds
       # other configs...
   }
   response = requests.get('https://api.zenrows.com/v1/', params=params)
   
   

   Find more details on our Wait Documentation on Wait For Selector Documentation.
5. Retry the Request

   Network issues or temporary failures can cause your request to fail. Implementing retry logic can solve this by automatically repeating the request.

   Python

   import requests
   from requests.adapters import HTTPAdapter
   from urllib3.util.retry import Retry
   
   # Define the retry strategy
   retry_strategy = Retry(
       total=4,  # Maximum number of retries
       status_forcelist=[429, 500, 502, 503, 504],  # HTTP status codes to retry on
   )
   # Create an HTTP adapter with the retry strategy and mount it to session
   adapter = HTTPAdapter(max_retries=retry_strategy)
   
   # Create a new session object
   session = requests.Session()
   session.mount('http://', adapter)
   session.mount('https://', adapter)
   
   # Make a request using the session object
   response = session.get('https://scrapingcourse.com/ecommerce/')
   
   if response.status_code == 200:
       print(f'SUCCESS: {response.text}')
   else:
       print("FAILED")
   

   You can learn more on our Python requests retry guide and Node.js Axios retry guide.
6. Get Help From ZenRows Experts

   Our support team can assist you if the issue persists despite following these tips. Use the Playground page or contact us via email to get personalized help from ZenRows experts.

   

For more solutions and detailed troubleshooting steps, see our Troubleshooting Guides.

Next Steps

You now have a solid foundation for web scraping with ZenRows. Here are some recommended next steps to take your scraping to the next level:

• Complete API Reference:
  Explore all available parameters and advanced configuration options to customize ZenRows for your specific use cases.
• JavaScript Instructions Guide:
  Learn how to perform complex page interactions like form submissions, infinite scrolling, and multi-step workflows.
• Output Formats and Data Extraction:
  Learn advanced data extraction with CSS selectors, output formats including Markdown and PDF conversion, and screenshot configurations.
• Pricing and Plans:
  Understand how request costs are calculated and choose the plan that best fits your scraping volume and requirements.

Frequently Asked Questions (FAQ)

How can I bypass CloudFlare and other protections?

To successfully bypass CloudFlare or similar security mechanisms, you’ll need to enable both js_render and premium_proxy in your requests. These features simulate a full browser environment and use high-quality residential proxies to avoid detection. You can also enhance your request by adding options like wait or wait_for to ensure the page fully loads before extracting data, improving accuracy.

Check out our documentation about the wait and wait_for params.

How can I ensure my requests don't fail?

You can configure retry logic to handle failed HTTP requests. Learn more in our guide on retrying requests.

How do I extract specific content from a page?

You can use the css_extractor parameter to directly extract content from a page using CSS selectors. Find out more in our tutorial on data parsing.

Can I integrate ZenRows with Python's Requests and BeautifulSoup?

Yes! You can use ZenRows alongside Python Requests and BeautifulSoup for HTML parsing. Learn how in our guide on Python Requests and BeautifulSoup integration.

Can I integrate ZenRows with Node.js and Cheerio?

Yes! You can integrate ZenRows with Node.js and Cheerio for efficient HTML parsing and web scraping. Check out our guide to learn how to combine these tools: Node.js and Cheerio integration.

How can I simulate user interactions on the target page?

Use the js_render and js_instructions features to simulate actions such as clicking buttons or filling out forms. Discover more about interacting with web pages in our JavaScript instructions guide.

How can I scrape faster using ZenRows?

You can scrape multiple URLs simultaneously by making concurrent API calls. Check out our guide on using concurrency to boost your scraping speed.