Over the last year I started experimenting heavily with Cursor, Claude and GPT for coding

Not because I wanted to become a “real developer,” but because I kept running into scraping problems where existing tools got me 80% moving forward when I was stuck with some scraping projects

And honestly, the biggest realization for me was this:

AI coding is insanely useful for people who understand scraping workflows better than programming itself

Most of what I do is SERP/data scraping, browser automation and scraping platforms thta aggressively rate limit or fingerprint traffic

Also I wouldn’t call myself a proper Python developer. My knowledge is actually really basic

But with AI-assisted coding I’ve managed to build:

• proxy rotation systems
• local APIs
• scraping helpers
• queue-based workers
• Telegram monitoring bots
• browser automation utilities
• retry/failure handling
• image processing tools
• account management workflows

What surprised me most is that the hard part usually isn’t the code itself.

It’s:

• structuring the architecture properly
• handling retries
• avoiding race conditions
• making sessions stable
• organizing proxies cleanly
• preventing spaghetti code
• figuring out where bottlenecks actually happen

That’s where AI became really helpful for me case

For example, instead of prompting:

>“make me a scraper”

I started prompting like:

>“Build a modular Python scraping system with rotating residential proxies, retry logic, queue workers, FastAPI endpoints, logging, rate limiting and browser session management.”

The difference in output quality is massive

One workflow I’ve been using a lot lately:

• FastAPI running locally
• scraping workers communicating through localhost
• rotating residential/mobile proxies
• anti-detect browsers for harder targets
• Telegram bots for monitoring/control
• queue systems handling retries automatically

Honestly, AI feels especially powerful for writing “glue infrastructure.”

Things like:

• proxy rotation layers
• retry logic
• task queues
• logging systems
• browser wrappers
• local APIs
• automation scripts
• monitoring tools

A lot of this used to feel inaccessible unless you were a serious engineer

Now solo scrapers can prototype infrastructure ridiculously fast

I’ve also noticed stable sticky sessions matter WAY more than raw proxy speed once you scale browser-based scraping or account workflows. Been testing some setups with a few proxy providers like node maven and brightdata recently for that reason because cleaner residential sessions + an antidetect browsers reduce random failures quite a bit actually

Biggest lesson so far: don’t try to build perfect systems immediately as this will not work, at least it didn't work for me

Now I mostly:

1. Build minimum working infrastructure
2. Stress test it
3. Fix bottlenecks as they appear

Way faster than overengineering from the start

Curious if other scrapers here are using AI coding tools heavily now for infrastructure/tooling?

Feels like the barrier to building custom scraping systems dropped massively in the last year.

I’ve been playing around with low-power e-ink projects lately and had an idea to create myself a useful calendar which looks cool, so I ended up building a Google Calendar display using an Inkplate board

The idea is simple:

• it pulls upcoming events from Google Calendar (about 1–2 weeks ahead)
• shows titles, times, and all-day events
• updates over Wi-Fi
• and just sits there like a paper calendar

What has been my favorite outcome of this whole project is that:

• it stays visible all the time (no backlight, no distractions)
• it only refreshes occasionally
• it runs for weeks (or longer) on a single charge

Hardware & setup

For this build I used:

• Inkplate 6 from Soldered(but this should work on other Inkplate boards too)
• Wi-Fi connection
• Google account (for API access)

The code is based on the Inkplate examples, so setup was pretty straightforward

Here's the end result:

I used older calendar data to test it

How I created this

There are 3 main pieces to get working, at least it worked for me:

1. Wi-Fi connection

Just add your SSID and password in the sketch that should look like this:

const char *ssid = "YOUR_SSID";
const char *password = "YOUR_PASSWORD";

2. Google Calendar API

You need to:

• create a project in Google Cloud
• generate an API key
• enable the Google Calendar API

Then drop the key into the code:

String apiKey = "YOUR_API_KEY";

3. Calendar ID + access

From Google Calendar:

• go to settings
• grab your Calendar ID
• make the calendar public (required for access)

​

String calendarID = "your_calendar_id@group.calendar.google.com";

4. Time sync

It uses NTP to keep track of time, so just set your timezone:

int timeZone = 2; // example: UTC+2

Why I actually really liked this project

A few things stood out while building this:

• didn’t have to fight the display much, Inkplate handles a lot for you
• power consumption is insanely low
• UI is simple but effective
• way less distracting than checking a phone or laptop

I want to share something that took me way too long to figure out (but currently I'm running a google scraping business that is really stable and employs 7 people) - I feel like this is one of really popular side hustles or actual business now try to open and do

A couple years ago I quit my software engineering job and started helping local businesses with Google stuff (Maps, reviews, visibility, basic presence)

At first it was super chill:
one client was a big one, then a couple of more reached out, then I had like 10+

And I thought, okay… this is it. Just get more clients and scale and it's a piece of cae

But not everything is easy as scaling a business to 20x

Not in an obvious way, but in a really annoying, subtle way:

• accounts getting flagged randomly
• actions not going through
• weird inconsistencies between accounts
• things working one day and not the next

I assumed it was like bad automation or just google being google

So I kept trying to “fix” everything else

The real issue was how everything looked from the outside

If you’re running multiple Google accounts, they don’t see: “oh this is a guy managing clients”

They see: “why are all these accounts coming from the same place?”

So if you're planning on doing this, you need ot make sure each account feels like a separate user

Not just different logins… but different environments overall (like using antidetect browsers + proxies)

I tried a bunch of proxy providers (not gonna lie, mostly picked based on price at first 😅)

At some point I stopped caring about “cheap” and just wanted something that works consistently

Once I found a setup that was stable (for me it ended up being NodeMaven, but honestly the bigger lesson is just: don’t cheap out on this part), things got… quiet.

Scaling the actual business

Once that was stable, scaling became straightforward:

What I changed:

• Standardized the process: Same workflow for every client and no more guessing
• Handled more accounts in parallel: Instead of babysitting each one, I could run multiple safely
• Took on more clients confidently: Before I was hesitant because things would break After and I knew the system could handle it
• Reduced time per client: Less debugging = more time actually delivering results

And that’s when I could finally:

• take on more clients without stress
• run multiple accounts properly
• actually scale without everything breaking

Looking back, that one change probably grew my business like 20x.

Not because it made me smarter or better at marketig
but because I removed the invisible bottlene

If I started again:

• I’d fix the infrastructure before trying to scale clients (this is really important i think)
• I wouldn’t waste time on cheap/unstable setups
• I’d test everything in small batches first (this part I actually did right)

Hi, I'm looking for like a basic proxy setup guide, maybe an article, a guide or someone who provides this, thanks

I'm not sure how many similar projects you are doing here, but over the past 9 months I’ve been experimenting with a few Arduino-based projects using e ink display / e paper display setups, mostly trying to build something low-power that can run for days or even weeks on battery

The main project I focused on was a battery-powered dashboard (weather + calendar + notifications) that updates every 10–15 minutes using WiFi (I strated doing this for my friend and I used this for my home actually in the end, and yes I made him one as well)

I went through a few different setups and hardware providers while trying to figure out what actually works well with Arduino without spending hours debugging display issues

Not affiliated with any of these, just sharing what worked for me and what didn’t

If you need help with a similar project please let me know (you can contact me in DMs)

Components I tested during this process

• ESP32 (Arduino IDE)
• Inkplate 10 (e ink display board)
• Waveshare 2.9” and 4.2” e ink displays
• BME280 (temperature + humidity)
• Li-ion battery (3.7V)

Also tested:

• Adafruit e-paper modules
• Seeed Grove components (for prototyping)

What I was trying to achieve with this project

• low power consumption (deep sleep most of the time)
• clean refresh without too much ghosting
• stable WiFi reconnect on wake
• minimal setup friction with Arduino libraries

What didn’t work great (initially)

1. Waveshare e ink display modules

These were the first ones I tried since they’re everywhere and probably the most popular modules out there

They do work, but for me:

• needed more manual setup (controller + wiring)
• some Arduino libraries weren’t plug-and-play
• took time to get consistent refresh working

Nothing wrong with them honestly, just more time spent getting everything stable

What worked better than expected

2. Inkplate from Soldered - was quite surprised with this one)

This is what I ended up using for the main dashboard.

Biggest difference:

• ESP32 already built in
• works directly with Arduino IDE
• power management already handled
• no need to wire display + controller separately

I didn’t have to “make the display work” first, it just worked, and I could focus on the logic, so this was a surprise for me how easily and really good it work

Other things I liked:

• fully open-source hardware
• good Arduino examples
• deep sleep + wake cycles behaved predictably
• low-power usage actually practical

3. Adafruit

Used this for a smaller status display project

What stood out:

• very clean Arduino examples
• libraries worked reliably
• documentation is probably the best out of all

Still more “component-level” than integrated:

• you’re wiring things yourself
• more setup than Inkplate

I would say this was a overall smooth sailing experience

3. Seeed Studio (Grove)

Used Grove modules for quick prototyping before I actually made the project

What I liked:

• super fast to connect everything
• less wiring headache
• easy to test ideas quickly

Not as focused on e ink display setups, but great for building prototypes

I’ve been testing a bunch of proxy providers over the past few months for automation and multi-account setups, and honestly… most of what you see on landing pages doesn’t hold up in real usage.. This has happend to me to many times

You know the usual stuff that providers say in their ads:

“50M+ IPs”, “99.9% uptime”, “perfect for any use case” and all that jazz (this of course isn't always the case)

Then you actually run traffic through them and things get a bit different from my experience

So instead of trusting marketing, I started focusing on a few things that actually matter:

First: pool quality > pool size
I’d take 1M clean IPs over 50M garbage ones any day.
If IPs are already flagged (Cloudflare, Spamhaus, etc.), it doesn’t matter how big the pool is. One bad IP can ruin an account.

Second: are they actually residential?
Some providers quietly mix datacenter IPs into “residential” pools. DO NOT TRUST THESE PROVIDERS WHO OFFER LIKE A proxy for 0.49$ per GB - IT'S A SCAM! No proxy is actually this cheap
That’s basically a guaranteed red flag for most platforms

Third: how they behave under real load
Ping doesn’t mean much form my experience. I’ve seen proxies with great latency completely fall apart when a browser opens 20+ connections and tries to load a JS-heavy page

How I tested (nothing fancy)

I didn’t use any fancy dashboards, just simple scripts:

• Light test (Axios) → check IP, GEO, ASN (to confirm it’s really residential)
• Heavy test (Puppeteer) → load a fingerprinting site and see how long it actually takes to render

That second one is where most proxies start struggling.

What surprised me

• Some providers looked decent in simple tests… then completely slowed down in real browser scenarios
• Lower ping didn’t mean better performance at all
• And yeah every single one still triggered WebDriver detection

That last one is important.
Proxies help with IP trust, but they don’t fix a bad browser fingerprint.

Quick ranking from what I tested

This is just my experience, not sponsored or anything:

1. NodeMaven
Probably the most consistent overall. Clean IPs and handled heavy pages way better than others. Felt like less babysitting was needed

2. Asocks
Solid budget option. Not amazing, but reliable enough if you’re scaling and watching costs

3. Floppydata
Works fine for lighter tasks. Starts struggling when things get heavier

4. Thordata
Could be useful for specific GEOs, but I wouldn’t rely on it for stable workloads

5. Bright Data
This one was frustrating. Expensive, requires KYC, and I ran into setup issues with their SSL handling. Didn’t feel worth the effort for my use case

Hope this helps

I’ve been building a few projects over the past 9 months (mostly low-power stuff with e ink display / e paper display / eink display setups), and I kept testing the products of 4 hardware providers trying to figure out what is best for my work

This isn’t sponsored or affiliated with anyone, just sharing what I’ve personally used and what worked (and didn’t) from a maker perspective

Also if you need any help with working on similar projects, feel free to DM me

Figured I’d put this together in case someone else is deciding between options

1. Soldered

I’ll start with Soldered because I ended up using them the most for anything related to e paper display projects

Project I used it for: I worked on a battery-powered home dashboard (weather + calendar + notifications) using their Inkplate 10 model

What stood out pretty quickly is that Inkplate isn’t just a raw eink display, it’s more of a complete system with:

• ESP32 already integrated
• power management handled
• libraries + documentation actually usable

What I really liked here is I didn’t have to spend time figuring out how to “make the display work”, bcs I could focus on the actual work

Other things I liked:

• fully open-source hardware (not just software)
• Arduino-compatible out of the box
• good examples that feel like real use cases
• low-power mode that’s actually practical

2. Waveshare

I’ve been using their products for almost 5 years now and was never disappointed in this

Project I used it for: I used a Waveshare e ink display for a smaller status screen project, where I wanted more control over the setup and didn’t mind doing the integration myself.

I specifically like this for a status project because:

• huge selection of eink display sizes and variants
• widely available and easy to source
• works with a lot of platforms (Raspberry Pi, Arduino, etc.)
• pretty well-documented for most common use cases

Still a great option if you want flexibility and don’t mind putting the pieces together yourself

3. Adafruit

Adafruit is probably the easiest to work with if you value documentation and polish

Project I used it for: A small eink display status screen for a Raspberry Pi server

What stood out for me here are:

• extremely well-written guides
• strong CircuitPython ecosystem
• consistent product quality
• beginner-friendly but still powerful
• good community + examples

They offer e paper display / eink display modules, but they’re more component-level

You’ll still do the integration yourself, which is good for learning but takes more time

4. Seeed Studio

In my experience, Seeed is more about modular systems, especially with Grove

Project I used it for: A quick environmental monitoring prototype using Grove sensors

What I liked from them:

• Grove system makes prototyping really fast
• wide range of modules
• easy to swap and test ideas
• decent balance between price and quality
• strong manufacturing capabilities

They also have e ink display options, but again, more modular

Works well if you like building systems piece by piece

If you have any questions, please let me know

I've seen this in one discussion on one forum and was interested in your opinion, but most providers give 1000MB per GB. Would it make a difference to you?