ToolScore
Data & Files 5 min read

The Real Cost of Scraping: Why 1,000 Working Pages Is Never 1,000 Requests

$0.40–$1.50 per 1K working pages the number that matters here

By Stan K.

Scraping pricing pages show $/GB and $/1K requests. Your real bill is set by retry rates, blocked-request billing and captcha triggers. The full math for 1M pages across a DIY stack, managed scraper APIs and enterprise unblockers.

Every scraping vendor prices something different. Proxy sellers quote $/GB. Scraper APIs quote $ per 1,000 requests. Captcha solvers quote $ per 1,000 solves. None of those is what you actually pay for — what you pay for is working pages, and the distance between “request” and “working page” is where the money goes.

Here is the full math, with verified numbers, so you can see where your budget actually lands.

Step 1: your retry multiplier

Raw scraping never succeeds 100% of the time. Measured success rates by target difficulty:

Target typeSuccess rateAttempts per working page
Clean e-commerce pages95–99%1.01–1.05
Financial sites85–95%1.05–1.18
LinkedIn, ticket brokers, guarded platforms75–90%1.11–1.33
Budget raw-proxy setups38–44%~2.5
Enterprise unblockers99%+~1.01

Two things jump out. First, the gap between a clean target and a guarded one is 10–30% more requests — a silent surcharge on every page. Second, a cheap proxy stack can be the most expensive option you own: at a 40% success rate you’re paying for 2.5 attempts per page, which erases any per-GB savings.

Step 2: the standard failure scenario

A realistic production configuration: 3–5 retries with exponential backoff, a 15% block rate, and captchas triggered on 10% of blocked requests. For 1,000,000 target pages:

  • ~1,175,000 raw HTTP requests (the 15% retry overhead)
  • ~15,000 captcha solves

That 17.5% overhead is the number that never appears on a pricing page but always appears on your invoice.

Step 3: the three architectures, priced

Same workload — 1M product pages/month, ~250 KB per page with assets, 15% block rate, 10% captcha rate:

DIY: proxies + solverManaged scraper APIEnterprise unblocker
Effective requests1,176,000 raw1,000,000 billed1,000,000 billed
Proxy / API cost~$882 (294 GB @ $3/GB residential)$399–499 (JS + proxy credits)$1,050–1,500 ($1.05–1.50 per 1K)
Captcha solver~$21 (17,600 solves @ $1.20/1K)includedincluded
Headless compute~$120 (2 cloud VMs)includedincluded
Total/month~$1,023$399–499$1,050–1,500
Cost per 1K working pages~$1.02$0.40–0.50$1.05–1.50

~2–2.5× — DIY costs that much MORE per working page than a mid-tier managed APIThe counterintuitive result

The DIY stack — the one that looks cheapest per GB — bills you for retries, captcha solves and two VMs that exist only to render other people’s JavaScript.

The plot twist: DIY wins without JavaScript

Strip JS rendering out of the equation — fetch raw HTML at ~30 KB instead of 250 KB through a browser — and the DIY picture flips completely:

  • Proxy usage drops from 294 GB to ~35 GB/month
  • The two Playwright VMs disappear
  • Total DIY bill: ~$125/month for the same 1M pages
1M working pages/month — total by architecture
DIY, no-JS (curl_cffi) ~$125
Managed scraper API $399–499
DIY: proxies + solver + VMs ~$1,023
Enterprise unblocker $1,050–1,500

A library like curl_cffi (MIT, impersonates real browser TLS fingerprints) can pull raw HTML through defenses that block plain requests/httpx — without a browser farm. That’s the setup we run ourselves, and it’s why the first question for any scraping project is not “which proxy vendor” but “does this site actually need JavaScript rendering?”

For the tools themselves — proxy pools, scraper APIs, captcha solvers with verified per-1K prices and their traps — see the Data & scraping hub.

How to read this for your own project

  1. Count working pages, not requests. Every comparison in this article is per 1K usable results.
  2. Measure your block rate on the real target before choosing an architecture. 5% and 40% block rates produce completely different optimal stacks.
  3. Check the free data first. A surprising amount of “scraping projects” are re-downloading data that governments and platforms already publish in bulk — we list nine verified free sources here.
  4. JS-heavy and DataDome/Turnstile-guarded targets are where managed APIs earn their price — they eat the retries, the captcha walls and the browser fleet, and bill only for delivered pages.

The vendors that charge for failed requests, the APIs that bill 404s as “successful,” and the solvers with no-refund policies are dissected in Where Scraping Money Quietly Leaks.

Numbers change — the math doesn't
Token prices and plan limits shift monthly; the structural arguments in this guide age slowly. For current per-tool numbers see the live rankings.