Scrape Google Search Results
Build a SERP monitoring script that tracks keyword rankings and extracts organic results using the Autom API.
Overview
In this playbook you will build a script that queries Google Search and extracts structured organic results — positions, titles, URLs, and snippets — for any keyword. A typical use case is rank tracking: run this on a schedule to monitor where your pages appear for target keywords.
The endpoint returns up to 10 organic results per page and supports pagination, country (gl) and language (hl) targeting.
Prerequisites
- An Autom API key — get one at app.autom.dev
- Install dependencies for your language:
pip install requestsSteps
Make your first search request
Call GET /v1/google/search with the q parameter and your API key in the x-api-key header.
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.autom.dev/v1/google/search",
headers={"x-api-key": API_KEY},
params={"query": "best python web scraping libraries", "gl": "us", "hl": "en"},
)
data = response.json()
print(data)Parse the organic results
The response contains an organic_results array. Each entry has position, title, link, snippet, domain, and source.
for result in data["organic_results"]:
print(f"[{result['position']}] {result['title']}")
print(f" URL: {result['link']}")
print(f" {result.get('snippet', '')}")
print()Handle pagination
Use the page parameter to fetch subsequent pages. The response includes a pagination object with has_next_page and next page number.
import requests
API_KEY = "YOUR_API_KEY"
QUERY = "best python web scraping libraries"
def fetch_all_results(query: str, max_pages: int = 3) -> list:
all_results = []
page = 1
while page <= max_pages:
response = requests.get(
"https://api.autom.dev/v1/google/search",
headers={"x-api-key": API_KEY},
params={"query": query, "gl": "us", "hl": "en", "page": page},
)
data = response.json()
all_results.extend(data.get("organic_results", []))
if not data.get("pagination", {}).get("has_next_page"):
break
page += 1
return all_results
results = fetch_all_results(QUERY)
for r in results:
print(f"[{r['position']}] {r['title']} — {r['link']}")Use the gl parameter to target a specific country (e.g. fr for France, de for Germany) and hl for the language of results. This is essential for accurate local rank tracking.
Model Context Protocol
Verbinden Sie Autom mit Claude Code, Cursor, Gemini CLI, OpenAI Developer Mode und anderen MCP-Clients über Ihren API-Schlüssel.
Monitor Google News
Set up a brand or keyword news alert pipeline that collects and deduplicates Google News articles in real time using the Autom API.