Block agent IP
Is anyone using Hermes for real estate specifically?
Hermes on a DigitalOcean droplet — Zillow/Redfin/Realtor all block the agent's datacenter IP.
How are people handling proxy/residential egress for consumer-site scraping?
Setup: Hermes on a DO droplet, Nous Portal Plus with Tool Gateway (Firecrawl) for web fetch.
Built a skill that needs data from Zillow / Redfin / Realtor / county assessor sites. Every primary source blocks the agent — CAPTCHA, PerimeterX challenges, "access denied." Agent diagnosed it cleanly: the droplet's DigitalOcean IP is a datacenter IP, and these sites aggressively block DC ranges. Confirmed it's the IP, not the fetch method, the same pages load fine from a residential browser context.
The skill behaves correctly (it refuses to fabricate and reports the block honestly), so this is purely an egress/IP problem, not a logic one.
Question for anyone who's solved this: what's the cleanest way to give Hermes a non-datacenter egress for web fetches?
Specifically:
Is anyone routing Tool Gateway / Firecrawl requests through a residential or ISP proxy? Provider recommendations that actually work against PerimeterX-protected sites?
Does the browser-automation toolset fare any better from a DC IP, or does it get blocked the same way (IP-level)?
Any pattern people prefer for consumer-site-heavy skills — proxy egress vs. a hybrid where the human supplies the protected data?
Not looking to do anything abusive, this is public property data for a personal valuation workflow, just need a non-blocked egress path. Appreciate any battle-tested setups.