Any automated workflows that you guys thought of and are using in your daily work?

Basically the title. If you have anything to share regarding automations, please do, looking to alleviate majority of manual day to day work.

reddit.com
u/jfurlong1977 — 5 days ago
▲ 115 r/ProxyEngineering+2 crossposts

so I accidentally learned how the entire proxy industry works, please tell me I'm not the only one

Started because I was overthinking this far too long and ended up reading about how free streaming apps on Samsung and LG TVs are literally enrolling your home IP into a residential proxy network while you watch Netflix or Prime on the other input. Like you click "agree" on some vague popup about "device resources" and your TV starts routing web traffic for whoever's paying for residential IPs that month. Up to 200GB or more, who knows?? All that while you're asleep. On your $2000 TV. Yall are fucking nuts if you think this is normal. And apparently that's just a thing that's happening and we should be all cool with it?? Boy whattahelly

Anyways, I went looking for who actually buys this traffic and why residential IPs are worth so much. And that's where it gets kinda funny. Turns out a huge chunk of the demand is scrapers trying to get past bot detection. Which, sure, makes sense, residential IP looks like a real human etc. Except I kept reading and finding that whole premise is like three years out of date. Cloudflare, Akamai, DataDome, the holy trinity. They're not primarily checking your IP anymore. They're fingerprinting the TLS handshake, the HTTP/2 frame ordering, your WebGL renderer string, how your canvas noise changes across sessions, whether your "mobile" session has gyroscope data. Your IP can be spotless and your Python requests library is still announcing itself on the first packet. Someone here on this sub literally tested this, datacenter IPs with proper TLS impersonation outperformed residential IPs with bare requests headers like 91% vs 60% on Cloudflare-protected site?? whaat?

So your TV is being used as a proxy node. The proxies are being sold to scrapers. The scrapers are getting detected anyway because the fingerprint gives them away. The TV thing was pointless to begin with basically. Am I looking to deep into this? Coz I feel like this shouldn't be normal, I mean the whole ordeal, not me looking into it. And then separately, completely disconnected from all this ordeal, some guy on Reddit figured out Spotify serves authenticated AAC files via CDN links that stay valid for 2 hours. Bro is going to get a DMCA notice fast if not already got it. What I'm trying to say is that Spotify already went through this exact thing. The tools are already built and already taken down. What else are we gonna have to deal with? And I'm talking about a small percentage of people that actually care, while the majority just clicks through all these popups without questioning a single thing.

Anyway I blocked proxyjs.brdtnet.com on my router, learned what JA3 fingerprinting is, and now I'm questioning every "agree" popup I've ever clicked. Normal tuesday, folks

reddit.com
u/jfurlong1977 — 13 days ago