Web Scraping Insider #8 | "ethical" residential proxy reckoning, free residential proxy tester, browser rewrite wave (CloakBrowser / Obscura / Camoufox)
Posted the latest Web Scraping Insider #8 if anyone here wants the full breakdown:
👉 https://thewebscrapinginsider.beehiiv.com/p/the-web-scraping-insider-8
Quick summary of what's inside:
⚖️ When "Ethical" Proxies Aren't Ethical
"Ethically sourced" has become the proxy industry's favourite marketing word. Almost no provider will show you which apps their residential IPs actually come from - no public partner list, no audit trail, no independent verification.
The last couple of weeks made that gap impossible to ignore:
- Spur Intelligence scanned 6,038 LG webOS + Samsung Tizen apps - proxy SDKs in 2,058 of them (42.5% on LG, 26.9% on Samsung)
- Bright Data's SDK enrolling always-on smart TVs as exit nodes, with consent buried in TV remote arrow-key navigation
- SuperBox streaming boxes (sold at major US retailers) shipping with dormant Popanet proxy software - routing third-party traffic through home connections with no meaningful consent
- FBI/IC3 now warning consumers that everyday devices are being silently turned into proxy nodes
None of those device owners meaningfully opted in. Yet those same residential IPs feed pools sold as "ethical."
Our take: "ethical" should be a claim you have to prove - published partner list, audit trail, who consented / in which app / when - not a landing-page adjective. My bet is the market moves there within the next year or two.
---
🔮 Proxy Tester: now benchmarks residential proxies too (free for you)
We expanded the ScrapeOps Proxy Tester beyond proxy APIs. It already benchmarks ~15 proxy-API-style providers against your exact target URL. Now it does the same for residential pools, so you can compare both side-by-side.
How it works: submit your URL → real requests through each provider → every config they expose gets tested → ranked by success rate + cost per successful request.
Residential is where marketing fluff runs deepest ("30M+ IPs", "99% success rates"). From what we've seen across billions of requests, CPM rarely correlates with performance on your actual target.
Try it: https://scrapeops.io/proxy-providers/tester/
---
🥊 The browser wars are back: people are rewriting Chromium itself
For a decade, scraping browser innovation meant automation libraries on top of Chrome (Selenium → Puppeteer → Playwright). The browser underneath was treated as a commodity.
That may be shifting. Two forces:
- Anti-bot reads deeper now - TLS, network stack, process behaviour - so runtime patches (playwright-stealth, undetected-chromedriver) break more often than they hold.
- Chrome is heavy at scale. Thousands of concurrent browser instances (or long-running AI agents) make a purpose-built engine attractive on cost + startup time.
Projects worth watching:
- CloakBrowser - Chromium fingerprints patched at the C++ source level, not JS injection. Drop-in Playwright/Puppeteer replacement. Claims 30/30 on public bot-detection suites.
- Obscura - Rust headless engine from scratch, CDP-compatible so Playwright still talks to it. Claims ~70 MB binary, ~30 MB RAM, near-instant startup vs Chrome's 200 MB+ / ~2s. (Self-reported, v0.1.0 - treat as experimental.)
- Camoufox - modified Firefox with C++-level fingerprint spoofing. Strongest headless evasion in independent tests we've seen. Proves this isn't only a Chromium story.
Stealth is moving below the automation layer. Most of these are young and several lean on self-reported numbers - don't rip out your production stack overnight - but the direction is worth tracking.
Bottom line: the residential proxy supply chain is getting scrutinised from every angle (smart TVs, factory hardware, federal warnings), the browser layer is getting rebuilt from scratch, and the boring work still wins - benchmark on your targets, measure cost-per-validated-payload, not vendor adjectives.
Happy to discuss specifics here - especially if you've benchmarked
— Ian (ScrapeOps)
