Trying to find a cloud-flare scraping solution
I am scraping a few TB with of data.
My experience with 5G proxies in real world application they operate 4G. Most providers throttle you from what I have seen.
Residential would be too much as that would add up fast.
Data centers from what I understand cloudflare curb stomps now.
The over all project is about 10TB
I got 3TB left to get. I was able to get a majority of it with my personal IP before captchas started hitting. From what I understand captcha solvers don't work unless you have a proxy.