The Real Cost of Scraping: Why 1,000 Working Pages Is Never 1,000 Requests
$0.40–$1.50 per 1K working pages the number that matters here
By Stan K.
Scraping pricing pages show $/GB and $/1K requests. Your real bill is set by retry rates, blocked-request billing and captcha triggers. The full math for 1M pages across a DIY stack, managed scraper APIs and enterprise unblockers.
Every scraping vendor prices something different. Proxy sellers quote $/GB. Scraper APIs quote $ per 1,000 requests. Captcha solvers quote $ per 1,000 solves. None of those is what you actually pay for — what you pay for is working pages, and the distance between “request” and “working page” is where the money goes.
Here is the full math, with verified numbers, so you can see where your budget actually lands.
Step 1: your retry multiplier
Raw scraping never succeeds 100% of the time. Measured success rates by target difficulty:
| Target type | Success rate | Attempts per working page |
|---|---|---|
| Clean e-commerce pages | 95–99% | 1.01–1.05 |
| Financial sites | 85–95% | 1.05–1.18 |
| LinkedIn, ticket brokers, guarded platforms | 75–90% | 1.11–1.33 |
| Budget raw-proxy setups | 38–44% | ~2.5 |
| Enterprise unblockers | 99%+ | ~1.01 |
Two things jump out. First, the gap between a clean target and a guarded one is 10–30% more requests — a silent surcharge on every page. Second, a cheap proxy stack can be the most expensive option you own: at a 40% success rate you’re paying for 2.5 attempts per page, which erases any per-GB savings.
Step 2: the standard failure scenario
A realistic production configuration: 3–5 retries with exponential backoff, a 15% block rate, and captchas triggered on 10% of blocked requests. For 1,000,000 target pages:
- ~1,175,000 raw HTTP requests (the 15% retry overhead)
- ~15,000 captcha solves
That 17.5% overhead is the number that never appears on a pricing page but always appears on your invoice.
Step 3: the three architectures, priced
Same workload — 1M product pages/month, ~250 KB per page with assets, 15% block rate, 10% captcha rate:
| DIY: proxies + solver | Managed scraper API | Enterprise unblocker | |
|---|---|---|---|
| Effective requests | 1,176,000 raw | 1,000,000 billed | 1,000,000 billed |
| Proxy / API cost | ~$882 (294 GB @ $3/GB residential) | $399–499 (JS + proxy credits) | $1,050–1,500 ($1.05–1.50 per 1K) |
| Captcha solver | ~$21 (17,600 solves @ $1.20/1K) | included | included |
| Headless compute | ~$120 (2 cloud VMs) | included | included |
| Total/month | ~$1,023 | $399–499 | $1,050–1,500 |
| Cost per 1K working pages | ~$1.02 | $0.40–0.50 | $1.05–1.50 |
~2–2.5× — DIY costs that much MORE per working page than a mid-tier managed APIThe counterintuitive result
The DIY stack — the one that looks cheapest per GB — bills you for retries, captcha solves and two VMs that exist only to render other people’s JavaScript.
The plot twist: DIY wins without JavaScript
Strip JS rendering out of the equation — fetch raw HTML at ~30 KB instead of 250 KB through a browser — and the DIY picture flips completely:
- Proxy usage drops from 294 GB to ~35 GB/month
- The two Playwright VMs disappear
- Total DIY bill: ~$125/month for the same 1M pages
A library like curl_cffi (MIT, impersonates real browser TLS fingerprints) can pull raw HTML through defenses that block plain requests/httpx — without a browser farm. That’s the setup we run ourselves, and it’s why the first question for any scraping project is not “which proxy vendor” but “does this site actually need JavaScript rendering?”
For the tools themselves — proxy pools, scraper APIs, captcha solvers with verified per-1K prices and their traps — see the Data & scraping hub.
How to read this for your own project
- Count working pages, not requests. Every comparison in this article is per 1K usable results.
- Measure your block rate on the real target before choosing an architecture. 5% and 40% block rates produce completely different optimal stacks.
- Check the free data first. A surprising amount of “scraping projects” are re-downloading data that governments and platforms already publish in bulk — we list nine verified free sources here.
- JS-heavy and DataDome/Turnstile-guarded targets are where managed APIs earn their price — they eat the retries, the captcha walls and the browser fleet, and bill only for delivered pages.
The vendors that charge for failed requests, the APIs that bill 404s as “successful,” and the solvers with no-refund policies are dissected in Where Scraping Money Quietly Leaks.