Web Scraping API
The Web Scraping API built for AI agents and pipelines
Scrape any URL to markdown, HTML, links and tables. Crawl a whole site as a background job. Map every URL a site publishes. Extract schema-defined records from pages that don't hand you a schema. One key, 1 credit per page — $0.30 per 1,000 — and from November 1, 2026, 5 for a page that has to be rendered, with no extra charge for output formats.
{ "searchTime": 0.17, "totalResults": 85000, "page": 1, "organic": [ { "position": 1, "title": "31.10. Connection Pools and Data Sources", "link": "www.postgresql.org/docs/…", "domain": "postgresql.org", "snippet": "For an environment without an application se…" }, { "position": 2, "domain": "medium.com", … }, { "position": 3, "domain": "stackoverflow.blog", … } ], "credits": 1}What is a web scraping API and what does it cost?
A web scraping API takes a URL and returns the page as clean markdown, HTML, links or structured records — rendering JavaScript when a page needs it — instead of raw code you parse yourself. Searlo charges 1 credit per page, $0.30 per 1,000 at the Scale pack; from November 1, 2026, a page that has to be rendered costs 5 credits.
What a real page costs is the number to compare. Firecrawl charges +4 credits (5x) for a JSON-formatted response instead of markdown. ZenRows charges 5 credits (5x) for JavaScript rendering, 10 (10x) for its premium option, and 25 (25x) for both together — exactly quantified, on their own pricing page. ScrapingBee and Scrapfly both say plainly that rendering, premium features and extraction cost more, without publishing the multiplier. Searlo publishes its one multiplier: from November 1, 2026, a page that had to be rendered costs 5 credits instead of 1, whatever output format you ask for, and the response's via field says which pages it applied to.
- Cost per call
- $0.30 per 1,000 (Scale pack, $74.99 one-time)— our pricing
- 100,000 pages
- $43.97 one-time (~$0.44/1K), for pages fetched without rendering— our pricing
- Page that has to be rendered
- 1 credit until October 31, 2026; 5 from November 1, 2026 (via: browser in the response)
- Free to start
- 3,000 credits, no card— our pricing
- Firecrawl JSON-format surcharge
- +4 credits/page (5x the base rate)— firecrawl.dev/pricing
- ZenRows JS render + premium surcharge
- 25 credits/page (25x the base rate)— zenrows.com/pricing
- Scrapfly per-request cost
- Depends on features enabled — no flat multiplier published— scrapfly.io/pricing
Third-party figures on this page last verified against the sources linked above. Prices change; if you find one of these stale, tell us and we will correct it.
Getting structured data off the web usually means stitching together several tools: one to fetch a page, one to render JavaScript, one to discover a site's URLs, one to crawl at depth, and one to turn a page into structured fields. Searlo's Web Data API is one key covering all of it. /scrape returns a single URL as markdown, plain text, raw HTML, links, images, headings, tables and the page's own JSON-LD in one call. /map lists every URL a site publishes, read from its robots.txt and sitemaps. /crawl walks a whole site as a background job with depth and path filters and a webhook on completion. /extract turns a field schema into filled-in records. /search-and-scrape runs a web search and fetches the pages it found, in one call — and for a single quick fetch, GET /search/webpage takes just a URL and returns clean markdown with metadata already parsed.
Every one of those costs 1 credit per page — $0.30 per 1,000 at the Scale pack — for a page fetched without rendering. From November 1, 2026, a page that needs a full browser render to load costs 5, and the response says which pages did. The section below sets that against what Firecrawl, ScrapingBee, Scrapfly and ZenRows charge extra for, with their own published numbers.
Extract is schema-in, records-out. Give it a field schema and one or more URLs, and it fills the schema from what the page already declares — schema.org JSON-LD, Open Graph, meta tags — before falling back to a language model only for whatever is left blank. A field the page never states comes back null and listed in missingFields rather than guessed, and you are billed per URL fetched, not per field or per model call.
Pages that render client-side are rendered automatically across every one of these endpoints (on accounts that have bought credits), so a React or Vue front end returns the same complete markdown as a static HTML page — you don't set a render flag ahead of time to find out which kind of page you're dealing with. Set render: false to keep every page at 1 credit.
Features
Scrape any URL
Markdown, plain text, raw HTML, links, images, headings, tables and the page's own JSON-LD — one credit, whatever formats you ask for (5 from November 1, 2026 if the page needs a browser).
Map a site's URLs
Every URL a site publishes, read from robots.txt and its sitemaps, falling back to on-page links when there is no sitemap.
Crawl whole sites as a job
Depth and path filters, a webhook on completion, robots.txt obeyed by default — billed per page as the job actually runs.
Extract schema-defined records
Give a field schema, get records back — sourced first from what the page declares, with an optional LLM fallback for what's left blank.
Search and scrape in one call
Run a web search and fetch the pages it found — one credit for the search, one per page actually scraped (5 from November 1, 2026 for a page that needs a browser).
Quick single-page fetch
GET /search/webpage takes just a URL and returns clean markdown with title, author and publish date — the lightweight mode for an agent's tool call.
Get started in one request
One credit, whichever formats you ask for. The same call works on a static blog post or a JavaScript-rendered app — rendering happens automatically when the page needs it, and from November 1, 2026 that page costs 5 credits.
curl -X POST "https://api.searlo.tech/api/v1/scrape" \
-H "x-api-key: YOUR_API_KEY" \
-H "content-type: application/json" \
-d '{
"url": "https://example.com/blog/how-we-ship",
"formats": ["markdown", "links", "tables"],
"onlyMainContent": true
}'import requests
r = requests.post(
"https://api.searlo.tech/api/v1/scrape",
headers={"x-api-key": "YOUR_API_KEY"},
json={
"url": "https://example.com/blog/how-we-ship",
"formats": ["markdown", "links", "tables"],
"onlyMainContent": True,
},
)
page = r.json()
print(page["metadata"]["title"])
print(page["markdown"][:500])
print(len(page["links"]), "links found")const res = await fetch("https://api.searlo.tech/api/v1/scrape", {
method: "POST",
headers: {
"x-api-key": "YOUR_API_KEY",
"content-type": "application/json",
},
body: JSON.stringify({
url: "https://example.com/blog/how-we-ship",
formats: ["markdown", "links", "tables"],
onlyMainContent: true,
}),
});
const page = await res.json();
console.log(page.metadata.title);
console.log(page.markdown.slice(0, 500));
console.log(page.links.length, "links found");Searlo vs Firecrawl
Firecrawl is the category's most visible product page and the clearest comparison. Both sell scrape, crawl, map and extract as one API — the difference is what a real page actually costs.
| Capability | Searlo | Firecrawl |
|---|---|---|
| Price per 1,000 pages (basic scrape) | $0.30 | $0.599–$3.20, depending on plan |
| 100,000 pages, total cost | $43.97 one-time (~$0.44/1K) for pages without a browser | $83/month (Standard, billed annually), recurring |
| Structured JSON output | Included, same 1 credit | +4 credits/page (5x) |
| Free credits to start | 3,000, one-time, no card | 1,000/month, only while on the free plan |
| Billing model | One-time credit packs, 1-year validity (new users) | Monthly subscription tiers |
| Site map + full crawl with webhook | Yes | Yes |
The credit-multiplier trap
Every competitor in this category quotes a headline per-1,000 rate, and every one of those rates is the price of the cheapest possible request — plain HTML, no rendering, no premium features, no structured output. The moment a real page needs any of those three things, which most pages do, the real cost is higher, and "higher" is a published, exact number on at least two vendors' own pricing pages.
ZenRows is the most explicit: JavaScript rendering costs 5 credits instead of 1 (5x), its premium option costs 10 credits (10x), and the two combined cost 25 credits (25x) — all stated directly on zenrows.com/pricing. Firecrawl charges +4 credits per page (5x) for a JSON, Question or Highlight-formatted response instead of plain markdown. ScrapingBee and Scrapfly both say plainly that JavaScript rendering, premium features and AI extraction cost more credits than a basic request, without publishing the exact multiplier on their pricing page — which is itself worth knowing before you commit to a plan sized on the headline rate.
Searlo publishes its one multiplier. A page fetched without rendering is 1 credit on /scrape, /crawl and /extract, whatever output format you ask for; from November 1, 2026, a page that had to be rendered costs 5, and each page's via field says which it was. /map stays 1 credit per call.
- ZenRows: 5x for JS render, 10x for its premium option, 25x for both (published)
- Firecrawl: +4 credits/page (5x) for JSON/Question/Highlight output formats
- ScrapingBee & Scrapfly: acknowledge render and premium surcharges, don't publish the number
- Searlo: 1 credit per /scrape or /map call (2 if you force a live fetch past the cache), 1 per page crawled, 1 per successful /extract URL; from Nov 1, 2026, 5 for a page that had to be rendered (published, and shown per page as via)
Extract: schema in, records out, with an honest LLM fallback
Most pages that matter already publish their own structured data — a product page carries schema.org JSON-LD with the price, an article carries the author and publish date, every page carries Open Graph tags. /extract reads what the page declares first, because that is exact, free to compute, and nothing a model can improve on.
What's left after that deterministic pass is optional model fallback — used only for fields the page did not declare, and only when a model is configured on the deployment. A field neither pass can fill comes back null and listed in missingFields, never guessed, and every field in the response records which of the two passes produced it so you can tell a value the site published from one a model inferred.
- JSON-LD, then Open Graph and meta tags, then unambiguous text patterns — model last, and optional
- Billed per URL successfully fetched, not per field and not per model call
- A URL that fails to fetch is never billed
- Every field's source is in the response — json-ld, meta, text, or llm
Crawl a whole site without babysitting a queue
A crawl is queued and returns a job id immediately — you poll it or supply a webhook, and credits are charged per page as the job actually runs, so a job capped at 500 pages that finds only twelve costs twelve. Path filters (includePaths, excludePaths) and a maxDepth keep a crawl from wandering into a site's entire history when you only want the blog or the docs; robots.txt is obeyed by default.
/map answers a narrower, cheaper question first: what exists on this site, before you pay to fetch any of it. It reads robots.txt for declared sitemaps, walks them — including sitemap indexes — and falls back to the homepage's internal links when a site publishes no sitemap at all. Run /map before a big /crawl and you know the real size of the job before you queue it.
- Job id back immediately; poll status or supply a webhook for completion
- includePaths / excludePaths as globs, maxDepth up to 10, concurrency up to 8
- robots.txt obeyed by default
- Cancel a running crawl any time — pages already fetched stay readable, and you stop paying
What teams build on the Web Data API
- RAG pipelines that need a page's clean content, not its raw HTML
- Product catalog and price-monitoring feeds from competitor sites
- Lead enrichment — company sites turned into structured firmographic fields
- Content migrations — mapping and crawling a whole site before a rebuild
- AI agent tool calls that need one quick page fetch mid-conversation
- Search-driven research pipelines that need the pages behind the results, not just the links
FAQ
Frequently asked questions
What is a web scraping API?
A web scraping API is an endpoint that takes a URL and returns the page as clean, structured content — rendered when the page needs JavaScript — instead of raw HTML you'd have to parse and maintain a parser for yourself.
How much does it cost to scrape 100,000 pages?
On Searlo, $43.97 one-time (the cheapest combination of credit packs reaching 100,000 credits), or about $0.44 per 1,000 pages, for pages that load without rendering; from November 1, 2026, each page that has to be rendered costs 5 credits. There is no subscription and no expiry pressure beyond the standard 1-year validity on new-user credits.
Does Searlo charge more for JavaScript-rendered pages?
Until November 1, 2026, no. From then, a page that had to be rendered (render: true, actions, or a page that returns no usable content without rendering) costs 5 credits instead of 1 on /scrape, /crawl, /scrape/batch, /extract, /search-and-scrape and /search/webpage, and the response says via: browser on those pages. Pages fetched without rendering stay 1, render: false keeps every page there, and /map stays 1 credit per call. A forced live fetch past the cache still adds 1. For comparison, ZenRows publishes 5x for rendering.
What's the difference between /scrape, /crawl and /map?
/scrape fetches one URL you already know. /map lists the URLs a site publishes, from its robots.txt and sitemaps, without fetching their content — useful for scoping a job before you pay to run it. /crawl fetches many pages starting from one URL, following links to a set depth, and is billed per page as it runs.
Does /extract use AI?
Only as a fallback. It reads a page's own schema.org JSON-LD, Open Graph tags and meta tags first — deterministic, exact, and free to compute — and only sends a model the fields that pass could not fill. A field neither step can find comes back null and listed in missingFields rather than invented.
Am I billed for a URL that fails to load?
No, for /extract — it is billed per URL successfully fetched, after the work runs, so a URL that could not be reached costs nothing. /scrape and /map are billed on the request itself, the same way every Searlo endpoint is.
Can I crawl a whole site and get notified when it's done?
Yes. POST /crawl returns a job id immediately; poll its status endpoint or supply a webhook URL and it fires when the crawl finishes. Credits are charged per page as the crawl runs, so the limit you set is a ceiling, not a quote.
Is there a free tier?
Yes — 3,000 free credits on signup, no card required, usable across every endpoint on this page.
How is a web scraping API different from a SERP or search API?
A search API answers "which pages exist for this query" — organic results, positions, titles, snippets. A web scraping API answers "what is actually on this page" once you have a URL, whether from a search result, a sitemap, or your own list. Searlo runs both under the same key; /search-and-scrape is the endpoint that combines them in one call.
Can I search the web and get the page content back in one call?
Yes — POST /search-and-scrape runs a web search and fetches the pages it found, returning each result with its scraped content attached. A page that cannot be fetched comes back with scraped: false next to the ones that worked, rather than failing the whole request.
From the team behind Searlo
We also build RenderKit — the browser-rendering API for screenshots, PDFs, and LLM-ready markdown. Same billing model, same developer-first API design.