u/HaeMGe

built a minimal, self-hosted alternative to airflow for people who just want to run a few scripts
▲ 11 r/ETL+1 crossposts

built a minimal, self-hosted alternative to airflow for people who just want to run a few scripts

was running a handfull of python scripts as cron jobs and it got annoying fast. no logs when something failed, dependency conflicts between scripts, no resource limits. looked at airflow, immediately closed the tab. a postgres db, redis broker, scheduler and webserver for 10 scripts felt insane.

so i built Fast-Flow. a pipeline is just a folder with a main.py. you git push, it syncs, it runs. no DAGs, no decorators, no image builds. dependencies get installed on the fly with uv and cached, so runs are fast even without pre-building anythign. each run happens in its own docker container or kubernetes job, your choice.

also has live logs, oauth login, encrypted secrets, cron scheduling. self hosted, no telemetry.

not trying to replace airflow for real DAG workflows, this is more for "cron but actually managable"

GPL-3.0, docs included: https://github.com/ttuhin03/fastflow

i made it, happy to answer question

u/HaeMGe — 4 days ago