Google Search Scraping API: How to Extract Rankings, Ads, and SERP Features

Google Search is the primary source of SERP intelligence for SEO performance monitoring, paid search analysis, and market visibility tracking. Rankings, ads, featured snippets, and local results change frequently and vary by location, device, and query intent. In this guide, we explain how to collect Google Search data, what SERP elements can be extracted, and how to do it reliably at scale using Nimble’s Web API.

What Data Can You Scrape from Google Search

Google Search exposes different data depending on query type, location, and SERP layout. Using Nimble’s Web API, search result pages can be rendered and parsed into structured JSON suitable for SEO platforms, analytics pipelines, and reporting systems.

Common Google SERP Data Points

• Organic result URLs, titles, and descriptions
• Organic ranking position per result
• Featured snippet presence and content
• People Also Ask questions and answers
• Paid ad titles, URLs, and ad position
• Top of page and bottom of page ads
• Local pack businesses, ratings, and addresses
• Knowledge panel entities and attributes
• Sitelinks associated with results
• Search query and detected intent
• Device and country context

This data is commonly used for rank tracking, competitive SEO analysis, ad coverage monitoring, and SERP feature optimization.

Why Use a Google Search Scraping API

Google Search results are highly dynamic and personalized based on geography, language, device, and query context. SERP layouts change frequently and differ across countries, even for the same keyword.

Key Challenges

• Location specific rankings and ads
• Device dependent SERP layouts
• Constant SERP feature experimentation
• JavaScript rendered elements
• High query volume requirements
• Need for normalized position tracking

Nimble’s Web API provides a consistent abstraction layer that handles access, rendering, extraction, and delivery at scale.

Need to accurately scrape Google search results by location? Check out our guide to Tracking Google Rankings Accurately Across Locations.

How to Scrape Google Search Data

Step 1: Choose the Google SERP Page Type

Google Search exposes different SERP structures depending on the query intent.

• Standard organic search results for keyword rank tracking
• Branded searches for knowledge panels and sitelinks
• Commercial queries for paid ad coverage
• Local intent queries for map pack visibility

Batch collection allows thousands of keyword queries to be processed concurrently.

Step 2: Send a Request to Nimble’s Web API

You send a request with the Google Search URL and query parameters. Nimble automatically handles the following.

• JavaScript rendering when required
• Session and access management
• Country and locale targeting
• SERP extraction and normalization

Example Python request for a Google Search results page

import requests
import base64
import urllib.parse

query = urllib.parse.quote("best noise cancelling headphones")

auth = base64.b64encode(b"YOUR_EMAIL:YOUR_API_KEY").decode()

response = requests.post(
    "https://api.webit.live/api/v1/realtime/web",
    headers={
        "Authorization": f"Basic {auth}",
        "Content-Type": "application/json"
    },
    json={
        "url": f"https://www.google.com/search?q={query}",
        "render": True,
        "parse": True,
        "country": "US"
    }
)

data = response.json()
print(data)

Step 3: Receive Structured Google SERP Data

Instead of raw HTML, the response contains structured JSON representing SERP elements and ranking positions.

Returned Fields

• Parsed organic results and rankings
• Paid ad blocks and placement
• SERP feature metadata
• Query and location context
• Optional raw HTML

Example response fields

{
  "entity_type": "organic_result",
  "position": 1,
  "type": "organic",
  "question": null,
  "title": "Best Noise Cancelling Headphones of 2025",
  "cleaned_domain": "example.com",
  "is_sponsored": false,
  "description": "An in-depth comparison of the best noise cancelling headphones available in 2025, including specs, pricing, and performance.",
  "snippet": "Compare the top noise cancelling headphones from Sony, Bose, and Apple, based on sound quality, battery life, and comfort.",
  "url": "https://www.example.com/best-noise-cancelling-headphones-2025",
  "displayed_url": "www.example.com › audio › headphones",
  "sitelinks_urls": [
    "https://www.example.com/headphone-reviews",
    "https://www.example.com/buying-guides",
    "https://www.example.com/comparisons"
  ],
  "sitelinks_first_url": "https://www.example.com/headphone-reviews",
  "current_page": 1,
  "next_page_url": "https://www.google.com/search?q=best+noise+cancelling+headphones&start=10",
  "other_page_urls_map": [
    {
      "page": 2,
      "url": "https://www.google.com/search?q=best+noise+cancelling+headphones&start=10"
    },
    {
      "page": 3,
      "url": "https://www.google.com/search?q=best+noise+cancelling+headphones&start=20"
    }
  ],
  "search_query_displayed": "best noise cancelling headphones",
  "search_total_results": "About 24,300,000 results",
  "shop_source": null,
  "shop_price": null,
  "shop_old_price": null,
  "shop_is_on_sale": false
}

Data can be returned in real time through the API response, or delivered asynchronously to cloud storage such as S3 or GCS for large scale SERP monitoring.

Nimble can also help you scale from one request to thousands. Check out our blog Scaling Google SERP Tracking From Dozens to Thousands of Keywords to learn how!

Best Practices for Google Search Scraping via API

Target location explicitly
Always specify country and locale to ensure consistent google api serp position tracking.

Separate desktop and mobile queries
SERP layouts and ranking behavior differ significantly by device type.

Track SERP features, not just links
Featured snippets, local packs, and ads influence visibility beyond organic rank.

Use batch collection for keyword scale
Batch thousands of keyword URLs to efficiently monitor large keyword sets.

Adjust collection frequency by keyword value
High value keywords benefit from more frequent SERP checks.

Getting Started with Nimble’s Web Scraping API

Create a Nimble account and generate an API key from the dashboard. The Web API uses a usage based pricing model aligned with query volume and rendering requirements. Documentation covers request parameters, batch workflows, and delivery options.

Sign up at https://app.nimbleway.com/signup
Read the docs at https://docs.nimbleway.com/

Continue Exploring Google SERP Data

If you are building a complete view of Google Search visibility, these guides expand on related use cases.

• How to Track Google Rankings Accurately Across Locations Learn why rankings differ by geography and how to collect location accurate SERP data.

• How to Monitor Competitors on Google Search Automatically Understand which domains compete with you across organic and sponsored results.

• Scaling Google SERP Tracking From Dozens to Thousands of Keywords See how teams move from manual checks to large scale SERP monitoring workflows.

‍

‍

‍