Skip to main content

[DRAFT] Serper search tools

Overview​

Serper is a Google Search API with a generous free tier and simple JSON responses. It’s handy for LLM agents that need web search without browser automation.

Why use Serper​

  • Lightweight REST API; no headless browser required.
  • Cheaper and simpler than scraping; predictable HTML-to-JSON mapping.
  • Works well with RAG pipelines and MCP/agent tool calls.

Sign up​

  • Create an API key at https://serper.dev
  • Note monthly free quota; add billing to increase limits.

Example (Python + requests)​

import os, requests

SERPER_API_KEY = os.environ["SERPER_API_KEY"]

def search(query: str):
    resp = requests.post(
        "https://google.serper.dev/search",
        headers={"X-API-KEY": SERPER_API_KEY, "Content-Type": "application/json"},
        json={"q": query, "gl": "us", "hl": "en"},
        timeout=15,
    )
    resp.raise_for_status()
    return resp.json()["organic"][:5]  # top 5 results

if __name__ == "__main__":
    results = search("langgraph vs langchain 2025")
    for r in results:
        print(r["title"], "-", r["link"])

Typical response fields​

  • organic: main results (title, link, snippet).
  • answerBox / knowledgeGraph / news / relatedSearches when applicable.
  • inlineImages for image search, videos for YouTube snippets.

Tips for LLM/RAG​

  • Limit results to top N and re-rank via embedding similarity to the user query.
  • Cache responses for popular queries to save quota.
  • Normalize URLs, strip tracking params, and fetch page content separately for grounding.
  • Add safety filters: blocklist domains, require HTTPS, and cap snippet length.

Comparison vs Playwright scraping​

  • Serper: fast, stable, predictable JSON; no JavaScript execution.
  • Playwright: full-page control, but heavier, slower, and costlier to run at scale.

Error handling​

  • Handle 429 (rate limit) with backoff and small sleep.
  • Validate organic exists; fallback to news or relatedSearches if empty.

References​