u/ahiqshb

Ok so this is gonna be a bit of a ramble but I need y'all to hear this. I work adjacent to the scraping/data space (not naming employer, y'all know how it is) and I always kind of just you know, accepted that "residential proxies" were this slightly grey but mostly fine thing. Like yeah obviously somebody's home IP is being used, somebody agreed to something somewhere, moving on. Then yesterday I see netnut is just gone. Not down, not maintenance page, straight up FBI DOJ IRS-CI seizure banner, Google and Lumen and Shadowserver all stamped on it too like it's a whole coalition operation. Like bruh, I have never in my life seen this on any website. So imagine the shocker lol. And I'm sitting there like wait since when did the IRS care about proxy IPs??? Turns out IRS-CI does financial crime investigations and it could be related to that, but still, seeing that logo on a proxy provider's main website??? Diabolical mate.

So I go searching what's going on and apparently it's tied to this thing called the Popa botnet, basically it's been running for like four years hijacking Android TV boxes (remember that Bright Data SDK thingy?? And Krebs traced a chunk of it back to actual NetNut infrastructure through some ex-employee's domain. Not some randoms, an actual former VP of R&D there. Google's own blog post says they think the network was over 2 million devices at one point and that a lot of those "different" proxy brands you see around are just NetNut wearing a different logo through resellers. I swear I seen a post talking about how there is one or two proxy providers sharing those IPs/reselling whatever. Makes sense now, people?? Am I the only one concerned here with a few other nerds? And to be fair I want to be fair here, Alarum (the parent company) came out and started calling it that a botnet is inaccurate, also said that there's real KYC and consent flows and misuse detection on their end. So it's not like this is some minor thing everyone agrees on, it's actively disputed and I think that matters, I'm not trying to just slam a company because Krebs wrote a scary headline, no. Make what you will out of it, but I things are rising to the surface, more and more often. Again it does make me think about how much of the "residential proxy pool" any of us are touching day to day is actually consented in a way a normal person would recognize as consent, versus consented in the "buried in paragraph 14 of an SDK terms screen" way. Like where's the line between legit residential network and just a nicer branded botnet, and how would any of us even know the difference from the outside.

Not trying to start a witch hunt on the whole industry, genuinely just spent my evening reading legal filings and threat intel blogs instead of sleeping like a normal person, but after Bright Data SDK shennanigans and now Netnut, boy, there's gonna be more stuff coming out. So I figured I'd share since I know a bunch of you here actually work with this infra daily and probably have opinions on these matters.

Sources if anyone wants to go check themselves.

Krebs on Security, the original Popa botnet reporting: https://krebsonsecurity.com/2026/06/popa-botnet-linked-to-publicly-traded-israeli-firm/

Google's threat intel writeup on the disruption: https://cloud.google.com/blog/topics/threat-intelligence/google-continued-disruption-residential-proxy-networks

Alarum's official response: https://alarum.io/alarum-technologies-responds-to-inquiry-into-residential-proxy-networks/

And divinetworks.com itself, which is also showing the seizure page now: https://divinetworks.com/

P.S Some time later I found that they mixed up the domains because .com is down, but not .io which is the main page of netnut.

If anyone's dealt with Hcaptcha while web crawling or running bots? I'm looking for something that can solve this without running a full Chromium/Firefox instance. The RAM and CPU overhead is not worth it for what I'm doing.

Ideally something that takes the sitekey and current URL and handles it without me having to babysit a whole browser session. I've seen a few repos floating around but most of them are either abandoned or don't work reliably anymore.

Anyone have something they actually use in production or at least tested recently? Would appreciate any help

This is a newbie friendly thread. Feel free to contribute if you think that your input will help the newcomers.

What mistakes did you do when using proxies for the first time? For Scraping? What adjustments you thought were worth the time. How is your approach differs from when you started to use the proxies?

Seeing how the TikTok fingerprinting post that blew up recently finally pushed me to write something I've been sitting on for a while too. Quite a few people DM'd me asking about my stack throughout several months, so here it is. Full breakdown, no provider shilling, honest and real discussion are welcome. (will make some spaces for easier reading).

That DesperateCoyote post nailed something most people in this space dance around: your IP being "clean" is basically table stakes now, not a competitive advantage. Platforms don't primarily care about your IP anymore. They care about whether your device behaves like a real device. I ran my own tests over few months across TikTok, LinkedIn, and a few e-commerce targets and the pattern was consistent. Sessions from "your beloved clean" residential IPs but running on a standard headless Chrome setup got flagged within 15 to 30 minutes on average. Sessions from a datacenter IP but running through Velum with proper sensor injection? Significantly longer survival rates. That's not a one-off result, thats the signal. The reason platforms moved this direction is actually pretty logical once you think about it from their engineering side. IP reputation databases are commoditized. Every half-decent anti-detect tool cycles through residential IPs so the defense shifted to the hardware layer, because that is harder to fake at scale.

The DesperateCoyote post described the fingerprint cluster score as a weighted confidence interval on "real human on a real device" which is accurate but undersells how granular it actually gets. WebGL renderer string consistency is one of the bigger ones, and not just that you have one but whether its consistent across sessions for the same "device." If your renderer string changes between sessions thats an immediate mark against you. Velum handles this quite well out of the box if you lock your profile, but it requires you to actually manage profile storage properly rather than spinning up fresh contexts every single run. Touch event timing is where things get genuinely impressive on the platform side. Real thumbs are sloppy, they drift, pressure varies, tap-to-tap timing has natural variance. Synthetic touch events are clean in a way that human fingers never are and the fix isnt randomizing your timing, its using distributions that actually match human motor patterns. This is where 8n8 has been doing interesting work, specifically the sensor emulation layer in their mobile profile configs. Battery API polling behavior is one I hadn't thought about much until this year. Real devices have battery levels that change over time in predictable ways, so a session that starts at 78% and is still at 78% four hours later is suspicious. Small signal but it adds to the cluster score. The timezone, DNS resolver, and behavioral gap is probably where most people get caught without realizing it. You can have a perfect German residential IP and a German timezone in your headers and still get flagged because your DNS resolver is resolving through a US-based server, or because your behavioral patterns like typing speed, scroll velocity, time-on-page don't match the demographic profile of a German user. Prismatic addresses this with what they call geo-behavioral profiles which bundle together not just the network layer stuff but the user behavior simulation parameters too. More holistic approach than anything else I'd seen and to be fair, it changed how I thought about the whole problem.

A lot of people in this space build scrapers and then bolt on some kind of orchestration layer as an afterthought. I was guilty of this for way too long, had scrapers running on cron jobs with zero retry logic, no rate management, and no session lifecycle management. Switching to Make for orchestration changed the output quality more than any proxy upgrade I made that year. When you manage workflows properly through something like Make you naturally end up with human-paced request timing because the automation has overhead between steps, session reuse instead of constant fresh session spawning which is itself a detection signal, proper error handling that doesn't hammer endpoints when things break, and actual audit trails so you can diagnose when and why sessions get flagged. The pacing thing sounds obvious but most people building direct scrapers optimize for speed and then add sleep() calls as an afterthought. Thats not the same as naturally paced requests coming from an orchestration layer doing real work between steps. Make's webhook and scheduling infrastructure also makes it way easier to implement robots.txt compliance and proper crawl-delay respect, which brings me to something I want to push back on slightly.

There was another post written on this sub that talks about big tech hypocrisy, and it is emotionally satisfying and honestly not wrong on the facts. OpenAI, Google, Meta et al absolutely did build on scraped data without permission and are now going after people for doing the exact same thing. The "fair use for innovation" asymmetry is worth being angry about. But I think it creates a mental model that's counterproductive for people actually building in this space. Whether scraping is ethical is a less interesting question than what the actual legal and technical constraints are and how to work within them. robots.txt is not legally binding in most jurisdictions but violating it is evidence of bad faith. The hiQ v. LinkedIn ruling established that scraping publicly available data generally isn't a CFAA violation, but courts look at whether you circumvented access controls and robots.txt plus explicit blocking counts toward that. Respecting crawl delays isn't just polite, its actually relevant to your legal exposure. ToS violations are a civil matter in most cases, not criminal. The Computer Fraud and Abuse Act has been interpreted narrowly by most courts post-Van Buren so violating a terms of service alone without credential-based access doesn't typically rise to unauthorized access under the CFAA. And even if you're legally clear, hammering a small site's server is just bad behavior. The Brendan Martin tutorial made this point well, one request per page, save locally, parse separately. Should be baseline for everyone and it usually isn't.

I am using several providers, as you would already guessed, however I will not say specific provider names because the landscape changes fast enough that recs go stale quickly, but the stack architecture I'm running now is ARM-based cloud phone environments for mobile targets (DesperateCoyote is right, this matters more than the proxy layer), Velum for profile management and sensor injection on desktop targets, 8n8 for mobile sensor emulation where Velum doesn't cover it, Prismatic for geo-behavioral profile bundling on targets where demographic coherence actually matters, Make for orchestration and scheduling, and local storage of raw HTML before parsing because yes, always, still. The shadowban rate on TikTok-adjacent targets dropped from roughly 55% to under 10% after moving to ARM environments. The proxy change alone, same environment just better proxies, moved it maybe 8 percentage points. The environment change moved it 37 points. That pretty much tells you where the leverage is.

There's a few things I haven't fully figured out and genuinely want input on. Canvas noise variance patterns specifically, DesperateCoyote mentioned they track how your noise changes not just that it exists, and I haven't found a clean way to implement dynamic noise variance that feels natural across sessions. Has anyone actually cracked this in a way that survives more than a few days? The accelerometer absence signal on mobile is another one. If you're running ARM cloud phones you have real accelerometers which is the obvious fix, but for desktop sessions claiming to be mobile is there a better approach than just not claiming to be mobile? Seems like the cleanest answer is just use the right tool for the target but I'm curious if anyone's found a workaround. Also been wondering about Make vs n8n for scraping orchestration specifically. I've been on Make for a while but n8n's self-hosted option is increasingly appealing for cost reasons on high-volume jobs. Anyone switched and have thoughts on the practical differences for this use case? And Prismatic's geo-behavioral profiles on non-Western targets, their coverage seems strong for US and EU but has anyone tested on Southeast Asian or LATAM targets where user behavior patterns are going to look pretty different? If I'm wrong about something say so and explain why, that's way more useful than a downvote.

I've been following the OpenAI lawsuits and the one detail I can't stop thinking about: a 19-year-old asked ChatGPT about mixing sedatives, it acknowledged the combo "could be risky", then gave him dosages anyway, added Benadryl to the recommendation, and told him to go lie in a dark room instead of seeking help. He died. Source. The Canadian case is somehow worse. OpenAI's own safety team flagged the shooter's account for "gun violence activity and planning" months before the attack and pushed to notify authorities. Management said no. Source. At some point "we're just a general-purpose tool" stops being a defense. Where that point is, that's what these trials are actually going to decide. Guardrails are coming whether the industry wants them or not. Every lawsuit forces a paper trail. And when harmful outputs become liability, the instinct is aggressive filtering, mandatory escalation triggers, activity logging with retention policies. Fine for consumer chat, however, for more tech enthusiasts its going to be brutal. Now the real risk for scraping and agentic workflows is over-correction. If "how do I access this data at scale" gets flagged the same way "how do I build a weapon" does, open-weights models win by default. It would make me want to just run it locally and skip the compliance layer entirely. The smarter play would be tiered access, stricter defaults for consumer products, more permissive behavior for verified API users with actual business context, but that requires product nuance, and right now OpenAI is in legal defense mode.

My bet is that we should expect more API friction over the next 12-18 months. Local models are about to get a lot more interesting.