Search API for RAG
Search API for RAG: fresh results and the page text behind them
A RAG pipeline needs two things from search: current results, and the text of the pages behind them, ready to chunk. Searlo returns both as JSON in one call, from $0.30 per 1,000 searches today, with 3,000 free credits to start.
{ "searchTime": 0.17, "totalResults": 85000, "page": 1, "organic": [ { "position": 1, "title": "31.10. Connection Pools and Data Sources", "link": "www.postgresql.org/docs/…", "domain": "postgresql.org", "snippet": "For an environment without an application se…" }, { "position": 2, "domain": "medium.com", … }, { "position": 3, "domain": "stackoverflow.blog", … } ], "credits": 1}What is the best search API for retrieval-augmented generation?
One that returns fresh results and the text of the pages behind them, in a shape your chunker can take without an HTML parser, at a cost that holds up at several searches per question. Searlo's POST /search-and-scrape does that in one call: a Google web search plus up to 10 of its result pages as markdown, for 1 credit plus 1 per page.
If your retriever only needs links and snippets, GET /search/web is 1 credit. Agents that connect through the MCP server get the same searches as TOON, about 63% fewer tokens than JSON.
- Search
- GET /search/web: up to 10 results with title, link, snippet and domain, 1 credit. About 1 s at the median, 2.5 s at p90 (September 2026)— docs
- Search plus page text
- POST /search-and-scrape: the results plus each page as markdown, text, HTML or JSON-LD. 1 credit for the search and 1 per page; from November 1, 2026, 5 for a page that has to be rendered— docs
- One retrieval
- A search plus 5 pages is 6 credits: $1.80 per 1,000 retrievals at the Scale pack rate— pricing
- Tavily
- $8.00 per 1,000 basic searches, pay-as-you-go— tavily.com/pricing
- Exa
- From $4 per 1,000 searches, plus $1 per 1,000 pages of contents— exa.ai/pricing
Third-party figures on this page last verified against the sources linked above. Prices change; if you find one of these stale, tell us and we will correct it.
What a search API has to do in a RAG pipeline
Retrieval runs search, fetch, chunk, embed, generate. A search API covers the first two: it finds the pages that answer the question today, and hands back their readable text without an HTML parser in between.
- Current results for news, prices and documentation
- Page text as markdown, ready to chunk
- A cost per question that survives 5 to 20 searches
- Latency that fits inside a user request
- Country and language control for regional answers
- Failed fetches that are not billed
Why RAG engineers use Searlo
Search and page text in one call
POST /search-and-scrape returns the results and each page as markdown, text, HTML or JSON-LD. A page that cannot be fetched comes back with scraped: false and is not billed.
Fresh Web Grounding
Web search returns in about 1 second at the median and 2.5 seconds at p90, measured on production traffic in September 2026. Fresh web context fits inside a RAG request.
TOON over MCP
Agents that call Searlo through its MCP server get TOON output: about 63% fewer tokens than the same results as JSON.
Prepaid credits
Pay once, from $3.99. No monthly fee eating into your RAG infrastructure budget; credits stay valid for 12 months.
Retrieve pages for RAG in one request
Search, then keep the pages that were fetched. There is no Searlo SDK to install: any HTTP client works, and a LangChain tool is a few lines.
curl -X POST "https://api.searlo.tech/api/v1/search-and-scrape" \
-H "x-api-key: YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{"q": "vector database benchmarks", "limit": 5, "formats": ["markdown"]}'import httpx
r = httpx.post(
"https://api.searlo.tech/api/v1/search-and-scrape",
json={"q": "vector database benchmarks", "limit": 5, "formats": ["markdown"]},
headers={"x-api-key": "YOUR_API_KEY"},
timeout=60.0,
)
docs = [
{"url": row["link"], "text": row["markdown"]}
for row in r.json()["results"]
if row.get("scraped")
]
# chunk and embed each doc["text"] with your splitter of choiceimport httpx
from langchain.tools import StructuredTool
def web_search(query: str) -> str:
"""Search the web and return the top results with their snippets."""
r = httpx.get(
"https://api.searlo.tech/api/v1/search/web",
params={"q": query, "limit": 5},
headers={"x-api-key": "YOUR_API_KEY"},
timeout=10.0,
)
return "\n".join(
f"{o['title']}\n{o['snippet']}\n{o['link']}"
for o in r.json().get("organic", [])
)
search_tool = StructuredTool.from_function(
func=web_search,
name="web_search",
description="Search Google for current information.",
)How Searlo Fits Your RAG Pipeline
User Query
User asks your AI application a question requiring current information
Search and fetch
Your app calls POST /search-and-scrape and gets the relevant results with their page text
Chunk & Embed
The markdown is ready for chunking, with no HTML to parse
Context Injection
Retrieved web context is injected into the LLM prompt alongside your domain knowledge
Generate Answer
The LLM generates a grounded, up-to-date response with the source URLs to cite
RAG-Optimized Pricing
| Provider | Search | Search + 5 pages |
|---|---|---|
| Searlo | $0.30 per 1,000 | $1.80 per 1,000 |
| Tavily | $8.00 per 1,000 (basic) | Page content is an option; see tavily.com/pricing |
| Exa | From $4 per 1,000 | From $9 per 1,000 ($4 + 5 × $1) |
FAQ
Frequently Asked Questions
What is the best search API for RAG pipelines?
One that returns fresh results and the page text behind them in one call, at a cost that holds up at several searches per question. Searlo's POST /search-and-scrape returns a Google web search plus up to 10 result pages as markdown, for 1 credit plus 1 per page, and web search answers in about 1 second at the median. If you need semantic search over a private index rather than live web results, a neural index such as Exa's is the other kind to consider.
Which search API is best for AI content generation?
For content that has to be current and cite its sources: a live web search with page text. Search with GET /search/web (1 credit, from $0.30 per 1,000), or fetch the pages too with POST /search-and-scrape, and pass the text and URLs to your model so it can quote and link them.
How does TOON format help with RAG?
TOON is the compact output of Searlo's MCP tools: abbreviated keys and no repeated fields, about 63% fewer tokens than the same results as JSON. It saves context when an agent reads search results directly. For chunking and embedding page text, use the markdown from /search-and-scrape.
Can I use Searlo with LangChain and LlamaIndex?
Yes, as a tool you define: there is no Searlo package to install. A LangChain StructuredTool or a LlamaIndex FunctionTool that calls the REST API is a few lines, as in the example above, and CrewAI takes the same function as a BaseTool. See the Integrations page for more examples.
How much does a RAG search pipeline cost with Searlo?
A search is 1 credit and each page fetched is 1 more, so a search plus five pages is 6 credits: $1.80 per 1,000 questions at the Scale pack rate, or $0.30 per 1,000 if snippets are enough. The free tier includes 3,000 credits, enough to build and test your pipeline (new users: 1-year validity, legacy users: unlimited).
Does Searlo support real-time search for RAG?
Yes. Searches return current Google results, and cache: false skips the cached copy when a query must be searched again now. That matters for RAG applications that need current information: news, prices, recent events or updated documentation.
From the team behind Searlo
We also build RenderKit — the browser-rendering API for screenshots, PDFs, and LLM-ready markdown. Same billing model, same developer-first API design.