r/ComplexWebScraping

I'm scraping google daily for competitor listings and ad placements, and the actual extraction isn't really the hard part anymore. the annoying part is captchas, blocks, proxy rotation and random changes that break the script every few days.

I'm looking at using a SERP API instead of maintaining all that in house. Teammate mentioned some platforms for handling the request and proxy side, but I'm curious how people here deal with this at scale. Anyone running a SERP API long term? mainly wondering where the cost starts making more sense than maintaining your own google scraper.