What's the best way to set up a proxy scraper using Python for small projects?
Hello. I've been working on a side project that pulls public data from social platforms like Facebook and X. I just need to track a few metrics for a small client, nothing that huge.
So far, the scraper works fine locally. But once I try running it consistently, it starts getting blocked pretty quickly. That's when I realized I probably need a proper proxy scraper setup. I spent a few nights reading about it here in this community. Ironically, instead of getting answers, I somehow ended up even more confused than when I started.
I'm not even sure if I should be managing rotating proxies myself or using a service that handles everything. There seem to be so many different approaches based on what I've come across so far, so it's hard to tell what actually makes sense for a smaller project.
For those who've built something similar in Python, how did you approach it? Did you implement proxy rotation yourself, or did you end up using an existing tool or service?