Login

u/Germond_

r/SideProject r/ReverseEngineering r/StremioAddons r/unRAID r/synology r/thepiratebay r/ProxyEngineering r/webdevelopment r/elfhosted r/PiracyArchive r/Servarr r/coolgithubprojects r/DataHoarder r/1337x r/PiracyBackup r/reactjs r/sonarr r/Lidarr r/javascript r/truenas r/WebScrapingInsider r/webscraping r/scrapingtheweb r/Softwarr r/github r/radarr r/ARR
TRAWL, a drop-in FlareSolverr replacement, is now available in Community Apps!
▲ 151 r/unRAID+2 crossposts

TRAWL, a drop-in FlareSolverr replacement, is now available in Community Apps!

Hey everyone! I'm the developer of TRAWL, a self-hosted web scraping engine and drop-in replacement for FlareSolverr/Byparr.

I originally built it for my own *arr stack after getting tired of slow solves and Cloudflare breakage. It uses a tiered approach: plain HTTP first, cached browser sessions when available, then a live Camoufox browser solve only when the site actually requires it.

It supports Cloudflare, Akamai and Imperva challenge detection, plus Turnstile, reCAPTCHA, hCaptcha and GeeTest. The existing FlareSolverr API is supported, so Prowlarr and Jackett users generally only need to change the service URL.

A community member has recently packaged TRAWL for Unraid, so installation no longer requires manually writing a Docker template:

  1. Open the Apps tab
  2. Search for Trawl
  3. Review the settings and click Install

Or simply get there by this link: https://ca.unraid.net/apps/trawl-1o4q23p06utr4h

The template runs standalone by default and already includes the required setting.
Redis is optional, but useful if you want solved browser sessions cached between requests.

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl

u/Germond_ — 4 days ago
▲ 71 r/StremioAddons

FlareSolverr kept killing half my indexers, so I built a drop-in replacement. Open source, self-hosted, and a lot faster.

If your setup (MediaFusion, Comet, Stremio-Jackett, anything through Jackett/Prowlarr) depends on FlareSolverr, you know the pain: half the time it flat out fails on modern Cloudflare, and when it does work it's slow. Byparr gets through more challenges, but it's a 4GB image that eats RAM and CPU, and it's still just as slow.

So I built TRAWL. It speaks the exact same FlareSolverr API, so you swap one URL and everything keeps working.

  • solves fresh Cloudflare in 4-15s, usually 2-8s once warm or with decent residential proxies, the other two take 11-18s when they manage at all
  • caches cookies per domain in Redis, so repeat requests return in under 500ms, and trackers are exactly that kind of workload
  • also solves Turnstile, reCAPTCHA v2, hCaptcha and GeeTest, no paid solver APIs
  • keeps a few warm browsers pooled instead of cold-starting one per request, stays light

docker compose up -d, then point your addon or indexer manager at http://trawl:8191, done. There's also a :baseline image for older hardware, supports kernels all the way down to 3.10, so even ancient NAS boxes are fine.

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl
Benchmarks: https://trawl.germondai.com/#compare

Fully open source, no telemetry, nothing leaves your box. Happy to answer questions, and if you hit any bugs, breakage, or want a feature, open an issue.

reddit.com
u/Germond_ — 1 month ago
▲ 5 r/DataHoarder

FlareSolverr kept failing on me, Byparr worked but was bloated - built something for grabbing stuff behind CF/hCaptcha

If you've got scripts/tools that need to get past Cloudflare (or hCaptcha) on some source sites, you've probably run into FlareSolverr just failing outright on newer challenges, and being slow even when it works. Byparr fixed the reliability side for me but it's bloated and still slow.

Built TRAWL to get Byparr-level (or better) success rates without the bloat - keeps a small pool of browser instances warm and reuses them instead of launching a fresh one per request, and once a domain's been solved once, repeat requests come back from a cached session almost instantly instead of re-solving every time. 2-6x faster overall than either alternative, and stays fairly light.

Also handles hCaptcha (checkbox auto-pass + audio-fallback for image challenges), Turnstile, reCAPTCHA v2, GeeTest v4, and Imperva/Incapsula sensor challenges, not just Cloudflare.

It's a drop-in for FlareSolverr's API, so anything already pointed at a FlareSolverr instance should just work by changing the URL. Also fine as a standalone CF-bypass proxy if you're just scripting fetches.

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl

reddit.com
u/Germond_ — 1 month ago
▲ 2 r/1337x

Prowlarr/Jackett constantly timing out when hitting 1337x? Here is what fixed it.

If you use Prowlarr to pull from 1337x, you've probably noticed that it throws constant Cloudflare blocks or 403 errors, causing your search to fail.

I got tired of my *arr stack failing because of this, so I built a drop-in replacement for FlareSolverr called TRAWL which is 3x faster, more reliable and actually solves captchas and challenges.

It uses a patched engine that actually bypasses the 1337x Cloudflare wall in a few seconds. More importantly, it caches the solution, so the next time you use search, it returns in ~500ms instead of spinning.

It's a 1:1 FlareSolverr replacement. Just spin up the Docker container and point Prowlarr to it.

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl

reddit.com
u/Germond_ — 1 month ago
▲ 4 r/coolgithubprojects

Wanna make your GitHub README pop? I built a massive, customizable icon API!

I was tired of boring, static SVGs and slow loading times on GitHub profiles, so I spent the time to build my own massive Icon API.

It’s designed to make your README look incredibly clean with zero effort. It's fully embeddable and dynamically adjustable purely via URL parameters - just change the URL string and the icon updates instantly.

The stats:

  • 6k+ unique tech brand icons
  • 13k+ total SVGs (comes in 4 distinct visual variants per icon)
  • Extremely lightweight and fast

If you want to ditch the slow, outdated services and make your GitHub profile look premium, give it a try.

Docs/API: https://icons.germondai.com
Repo: https://github.com/germondai/icons

(If you are interested to self-hosting: ghcr.io/germondai/icons:latest)

u/Germond_ — 2 months ago
▲ 19 r/synology

Tired of FlareSolverr breaking your *arr stack downloads? Here’s a faster drop-in replacement for your Synology!

If you're running Prowlarr or Jackett on your Synology to fetch stuff for Sonarr/Radarr, you've probably run into the endless loop of indexers failing because of Cloudflare blocks. The usual fix is FlareSolverr, but let's be honest, it's slow (15-18s per solve), constantly breaks, and eats a ton of RAM on a NAS.

I got tired of babysitting it on my own homelab, so I built TRAWL as a drop-in replacement. You literally just swap the FlareSolverr URL in your *arr stack to the new TRAWL container, and it just works.

Why it's better for a NAS:

  • Faster: Solves Cloudflare in 4-15s (uses Camoufox to trigger CF's fast-path)
  • Saves CPU/RAM: It hooks into Redis. The first request to a domain takes a few seconds, but repeat requests return in ~500ms without spinning up a browser at all
  • Actually solves captchas: Handles Turnstile, reCAPTCHA, hCaptcha out of the box

Synology Setup Note: If you're using the Synology Container Manager on some older hardware, I highly recommend using the :baseline Docker image tag instead of :latest. It includes specific fixes for older Synology kernels so you won't hit any of those weird module resolution errors.

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl
Docker image: ghcr.io/germondai/trawl:baseline

Happy to answer any questions if anyone wants to migrate their setup!

u/Germond_ — 2 months ago
▲ 4 r/webdevelopment+2 crossposts

Replace skillicons.dev in your READMEs - 6,150 icons including everything released in the last 2 years

skillicons.dev has been basically unmaintained for a while now. Missing bun, drizzle, hono, cursor, windsurf, claude, turso, neon, convex, htmx - you know the list. The GitHub issues have been sitting unanswered for months.

Found a free replacement: icons.germondai.com

Same one img tag. 18x more icons. Sub-100ms responses.

![icons](https://icons.germondai.com/icons?i=typescript,react,bun,tailwindcss,docker&theme=dark)

Per-icon control (variant, color, background, radius per icon, all in one URL):

![icons](https://icons.germondai.com/icons?i=typescript:original:3178c6,react:mono:61dafb,bun:plain:fbf0d9&theme=slate&size=56)

6,150 icons. MIT licensed. Free to use.

GitHub: https://github.com/germondai/icons (a star means a lot)

u/Germond_ — 1 month ago
▲ 15 r/elfhosted

I built a FlareSolverr replacement that's 3× faster and actually solves captchas!

Been running FlareSolverr for a long time for my *arr stack and got tired of the 11-18s solve times and constant breakage. Built TRAWL as a drop-in replacement!

Key differences:

  • Cloudflare solves in 4-15s (vs 11-18s) - uses a fresh Camoufox Firefox context which triggers CF's fast-path evaluation
  • Cached repeat requests return in ~500ms via Redis - after the first solve, the same domain is instant
  • Actually solves in-page captchas: Turnstile (shadow DOM click), reCAPTCHA v2 (free Google STT audio), hCaptcha (auto-pass), GeeTest v4
  • 4-tier execution: plain HTTP → cached session → live browser solve → residential proxy. You pay the full browser cost only when you have to
  • Custom headers support - pass Authorization, Referer, Origin through all 4 tiers including browser
  • FlareSolverr v2 compatible - change one URL in Prowlarr/Jackett, nothing else

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl

Happy to answer questions. Still early but it's been running stable on my homelab and no issues so far.

reddit.com
u/Germond_ — 2 months ago
▲ 18 r/radarr

I built a FlareSolverr replacement that's 3× faster and actually solves captchas!

Been running FlareSolverr for a long time for my *arr stack and got tired of the 11-18s solve times and constant breakage. Built TRAWL as a drop-in replacement!

Key differences:

  • Cloudflare solves in 4-15s (vs 11-18s) - uses a fresh Camoufox Firefox context which triggers CF's fast-path evaluation
  • Cached repeat requests return in ~500ms via Redis - after the first solve, the same domain is instant
  • Actually solves in-page captchas: Turnstile (shadow DOM click), reCAPTCHA v2 (free Google STT audio), hCaptcha (auto-pass), GeeTest v4
  • 4-tier execution: plain HTTP → cached session → live browser solve → residential proxy. You pay the full browser cost only when you have to
  • Custom headers support - pass Authorization, Referer, Origin through all 4 tiers including browser
  • FlareSolverr v2 compatible - change one URL in Prowlarr/Jackett, nothing else

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl

Happy to answer questions. Still early but it's been running stable on my homelab and no issues so far.

reddit.com
u/Germond_ — 2 months ago
▲ 159 r/Softwarr+11 crossposts

TRAWL: Self-hosted scraping engine — bypasses any JS challenge & captcha: Cloudflare, Turnstile, reCAPTCHA, hCaptcha, GeeTest. FlareSolverr & Byparr alternative and drop-in replacement for your *arr stack.

github.com
u/Germond_ — 1 month ago
▲ 235 r/Servarr+3 crossposts

I built a FlareSolverr replacement that's 3× faster and actually solves captchas!

Been running FlareSolverr for a long time for my *arr stack and got tired of the 11-18s solve times and constant breakage. Built TRAWL as a drop-in replacement!

Key differences:

  • Cloudflare solves in 4-15s (vs 11-18s) - uses a fresh Camoufox Firefox context which triggers CF's fast-path evaluation
  • Cached repeat requests return in ~500ms via Redis - after the first solve, the same domain is instant
  • Actually solves in-page captchas: Turnstile (shadow DOM click), reCAPTCHA v2 (free Google STT audio), hCaptcha (auto-pass), GeeTest v4
  • 4-tier execution: plain HTTP → cached session → live browser solve → residential proxy. You pay the full browser cost only when you have to
  • Custom headers support - pass Authorization, Referer, Origin through all 4 tiers including browser
  • FlareSolverr v2 compatible - change one URL in Prowlarr/Jackett, nothing else

Website: https://trawl.germondai.com
Docs: https://docs.trawl.germondai.com
GitHub: https://github.com/germondai/trawl

Happy to answer questions. Still early but it's been running stable on my homelab and no issues so far.

reddit.com
u/Germond_ — 2 months ago