u/klausblade

I'm building an video platform and right now it lives in 4 separate repos:

1. Main web app — React (TanStack Start), deployed on Vercel
2. Admin dashboard — also React/TypeScript, internal tool
3. API backend — Python FastAPI + a Redis worker, deployed on AWS/Railway
4. Video pipeline — Python, runs on AWS Lambda

They all talk to the same Supabase (Postgres) database and Redis, and they call each other. The database migrations live in the main web app repo, but the Python services read the same tables.

The pain points I'm having:

• When I change the database schema, I have to update types/models in multiple repos by hand and keep them in sync.
• A single feature often means opening PRs in 2–3 repos at once.
• Shared config (env vars, API contracts) drifts between repos.

My question: is it worth moving all of this into one monorepo, even though it's mixed TypeScript + Python with completely different deploy targets (Vercel, AWS Lambda, ECS)? Or is the mixed-language, mixed-deployment situation exactly when you should NOT do a monorepo?

If you've done this — did tools like Turborepo/Nx/Pants handle the Python side okay, or did you just use a plain monorepo with separate CI workflows per folder? Any regrets either way?

Solo dev / small team, so I care more about "less friction day to day" than big-org tooling.

Tech Stack

• FastAPI
• Upstash Redis
• AWS ECS (workers with autoscaling)
• Supabase (persistence + realtime)

Flow

1. User submits a prompt.
2. API creates a job and redirects the user to /chat/:id.
3. Job is pushed to Redis.
4. ECS workers pick up jobs and process them.

Each job has multiple stages:

• Stage 1: Call Claude, stream results to frontend, then wait for user approval.
• Stage 2-4: More Claude/Gemini calls and processing.
• Total runtime is usually 8-10 minutes.

For realtime updates, workers write streaming chunks directly to Supabase. The frontend subscribes to the job data, so users see updates live. If they refresh the page, they reconnect and continue from the latest persisted state.

For recovery, I store checkpoints after every stage. If a job becomes stale (e.g. no updates for 15 minutes), a recovery process checks whether it's still running and resumes it from the last checkpoint.

Current Problem

Each worker processes up to 3 jobs concurrently.

Most of the workload is async (waiting on Claude/Gemini APIs). So when Job A is waiting for an LLM response, the worker starts Job B and Job C.

The good part is that worker utilization is higher. The bad part is that individual jobs take significantly longer to complete because multiple jobs are competing for resources and API calls at the same time.

I'm wondering:

1. Is running multiple jobs per worker the right approach for this type of workload?
2. Would you instead run 1 job per worker and scale ECS horizontally?
3. Is there a better pattern for orchestrating long-running multi-stage workflows with human approval checkpoints?
4. Does my recovery strategy sound reasonable, or is there a more robust way to handle stuck jobs, worker crashes, and retries?

Curious how others would design this system.

Tech Stack

• FastAPI
• Upstash Redis
• AWS ECS (workers with autoscaling)
• Supabase (persistence + realtime)

Flow

1. User submits a prompt.
2. API creates a job and redirects the user to /chat/:id.
3. Job is pushed to Redis.
4. ECS workers pick up jobs and process them.

Each job has multiple stages:

• Stage 1: Call Claude, stream results to frontend, then wait for user approval.
• Stage 2-4: More Claude/Gemini calls and processing.
• Total runtime is usually 8-10 minutes.

For realtime updates, workers write streaming chunks directly to Supabase. The frontend subscribes to the job data, so users see updates live. If they refresh the page, they reconnect and continue from the latest persisted state.

For recovery, I store checkpoints after every stage. If a job becomes stale (e.g. no updates for 15 minutes), a recovery process checks whether it's still running and resumes it from the last checkpoint.

Current Problem

Each worker processes up to 3 jobs concurrently.

Most of the workload is async (waiting on Claude/Gemini APIs). So when Job A is waiting for an LLM response, the worker starts Job B and Job C.

The good part is that worker utilization is higher. The bad part is that individual jobs take significantly longer to complete because multiple jobs are competing for resources and API calls at the same time.

I'm wondering:

1. Is running multiple jobs per worker the right approach for this type of workload?
2. Would you instead run 1 job per worker and scale ECS horizontally?
3. Is there a better pattern for orchestrating long-running multi-stage workflows with human approval checkpoints?
4. Does my recovery strategy sound reasonable, or is there a more robust way to handle stuck jobs, worker crashes, and retries?

Curious how others would design this system.