u/tecsapling

Hey everyone,

I’m currently building a small micro-SaaS utility (Honestify.me) to collect anonymous professional feedback. To prevent the usual toxic garbage you get with anonymous text boxes, I hooked up gemini-2.5-flash to act as an HR mediator—it intercepts the payload, scores the toxicity, strips the insults, and saves the constructive critique to Postgres.

The Problem: My intake pipeline is completely serverless. When I initially wrote the API route synchronously, the Next.js backend was holding the HTTP connection open while waiting for Google's AI Studio to return the JSON response.

Submissions were taking 7 to 8 seconds. The UX was terrible, users thought the "Submit" button was broken, and I was dangerously close to Vercel’s 10-second serverless timeout limit.

The Fix (Asynchronous Decoupling): Instead of setting up a heavy SQS/Kafka queue for a simple side project, I restructured the API route to decouple the database insertion from the LLM processing using background execution.

Here is the flow I settled on:

1. Security Gates: Upstash Redis (rate limiting) + Cloudflare Turnstile (bot check).
2. Immediate DB Insert: Prisma creates a PENDING record with the raw text.
3. Release the Client: Instantly return NextResponse.json({ success: true }) -> UX is now sub-300ms.
4. Background Envelope: Use Vercel's waitUntil (or an unawaited async IIFE) to ping Gemini 2.5, parse the JSON, and update the Postgres row to COMPLETED or REJECTED in the background.

Here is a simplified version of the route pattern:

import { NextResponse } from 'next/server';

import { waitUntil } from '@vercel/functions';

import prisma from '@/lib/prisma';

import { GoogleGenerativeAI } from '@google/generative-ai';

export async function POST(req: Request) {

// ... [Upstash & Turnstile validation here] ...

const { message, username } = await req.json();

// 1. Instantly save the raw state

const feedback = await prisma.feedback.create({

data: { message, status: 'PENDING', isAiRewritten: false }

});

// 2. Fire the background worker

waitUntil(

(async () => {

try {

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY!);

const model = genAI.getGenerativeModel({ model: 'gemini-2.5-flash' });

// ... Prompt logic enforcing JSON output (PASS, REWRITE, BLOCK) ...

const response = await model.generateContent(prompt);

const aiData = JSON.parse(response.text());

// 3. Update DB based on AI triage

if (aiData.action === 'REWRITE') {

await prisma.feedback.update({

where: { id: feedback.id },

data: {

message: aiData.rewritten_message,

originalMessage: message,

isAiRewritten: true,

status: 'COMPLETED'

}

});

}

} catch (error) {

console.error("Background AI processing failed:", error);

}

})()

);

// 4. Instantly release the client UI (< 300ms latency)

return NextResponse.json({ success: true, id: feedback.id });

}

The attached screenshot shows the result of the background pipeline catching a toxic "spaghetti code" rant and rewriting it into a professional critique.

If anyone is trying to run non-blocking LLM moderation on Vercel without paying for external queue services, this pattern has been incredibly stable for me so far. You can test how the frontend handles the immediate response on the live domain (honestify.me) if you want to try breaking the AI edge cases.

How are you guys handling long-running LLM tasks on the edge? Are you defaulting to background functions, or pushing everything to specialized queue services?