u/Common_Dream9420

I am building FetchSandbox because I kept running into the same problem: the only way to test if an agent-built workflow actually works is to connect a real account. Real emails send. Real calendar invites go out. Real Slack messages drop. All before you even know if the logic is right.

The part that bothered me more was receipts. A success status doesn't tell you if the app did the right thing. So our sandbox goes deeper, it checks the resulting state, it replays webhooks, it catches whether a retry fires a duplicate charge or a duplicate email you'd never see in a test.

Right now I'm trying to figure out something: which workflow is actually the hardest to safely test before going live? The incoming email to calendar booking to Slack confirmation is the one I keep hearing about, but I don't know if that's just the loudest voice in the room.

If you've built something like this and had to just "yolo and connect real accounts," genuinely curious what broke.

Been thinking about what “author must prove it works” actually means for background jobs. CI green and linter happy doesn’t answer the retry question.

Here’s the scenario I keep coming back to: a job commits a database row, then crashes before it sends the acknowledgment back to the queue. The broker replays it. Without a deduplication check, that can create a duplicate effect. A successful first-run test never surfaces this, the happy path looks clean.

Some of the discussions here describe reviewers reconstructing the failure path themselves rather than requiring the author to show it upfront. That shifts ownership the wrong way.

What does your team actually require? Does the author have to show an executed failure/retry check and confirm it catches the old broken behavior, or is that left to the reviewer to reconstruct?

(Disclosure: I’m building FetchSandbox, a verification tool, so I lean toward execution evidence. Just keeping that visible.)

anyone building ai automaton workflows with looplabs ?

Been heads down building FetchSandbox for the past 3 months. Ran a 30 vs prior-30 comparison in PostHog this week. Numbers moved so sharing them, but also being precise about what they actually are.

- MCP tool calls: 422, up 49% from 283

- Tracked MCP identities: 31, up 138% from 13

- Portal-entry identities: 307, up 97% from 156

- Workflow-view identities: 70, up 59% from 44

These are PostHog analytics identities and events, not verified unique developers or new signups. The 138% looks impressive but it's not a user growth number, wanted to say that upfront.

The gap we're actually focused on now: "an agent called our MCP" and "the agent actually configured the app, exercised it, and got a clear verification result" are very different things. First one is easy to track. Second is what actually matters and harder to measure.

For people building with Lovable, Replit, or other coding agents: what evidence would actually convince you that an integration was properly tested before launch?

We've been building a platform where you submit your favorite posts and we stream them back to you. You can also subscribe to web events so you're not missing things you care about.

wondering if anyone here is already scraping the web for this kind of thing.

Do you use a tool for it, or roll your own? Would love to know what the workflow actually looks like before we go further down this road.

if u are building and deploying agents in prod which takes actions on behalf on consumers … how are u validating agent actions and their execution runtime?

We’ve been building FetchSandbox

for nearly 2.5 months, and one prompt in Lovable

completely changed the product’s direction.

Here’s the prompt that made the difference:

>"Build a gym membership app with Paddle subscriptions, Resend for emailing customer invoices, and resilient handling for duplicate webhooks, auth failures, rate limits, and payment invariants, tested against FetchSandbox twins. Build on Lovable’s stack (React, TypeScript, Lovable Cloud) without Next.js or a framework."

This single request pushed us to refine our approach, improve resilience, and streamline our tech stack. What do you think? How would you refine this further?

https://preview.redd.it/kyjzxxi334rh1.png?width=1080&format=png&auto=webp&s=4e9210870b08b7ec60fb228f0030010d119265b9

Connected FetchSandbox to Lovable last week and ran a real app through it. Gym membership app for a local trainer, Paddle payments and Resend emails.

No real keys, no actual charges. Injected duplicate webhooks, retry edge cases, auth failures. Found 4 bugs before the trainer ever saw it.

The thing I kept thinking about after: the agent that built the app has no proof the integrations actually hold. Happy path passed, it looked right, preview was clean. That's probabilistic confidence, not verification. One duplicate webhook event in prod and the whole thing unravels silently.

What we built is a deterministic verification layer for exactly this. Real failure injection, real replay, actual pass/fail on edge cases. Not "the LLM said it looks fine."

If you're building on Lovable, Replit, or Base44 and your agent is executing real payments and webhooks on behalf of users, how are you actually validating it did what it was supposed to do?

https://fetchsandbox.com/ai-app-builder-integration-testing

curious do base44 provides any tools or connectors that i can connect to validate webhooks or messy production flows before i take my app to productionize?

<disclaimer maybe some ppl find long post but just trying to understand pain is real or not …>

I have been talking a lot lately from devs and eng managers: review queues are backed up, the diffs are huge, and nobody is confident the code is actually safe to ship.

It makes sense. Agent-generated code moves fast. But the review process didn't evolve with it. You're still squinting at diffs, still relying on someone to mentally simulate what the agent did, and still hoping the happy path covers enough.

A few questions I'm genuinely curious about:

1/How are you actually verifying agent-written code before it merges? Is it still just human review, or have you layered something else in?

2/Are you running into edge cases that passed review and broke in prod anyway?

3/And the bigger one: do you trust that your current testing setup catches behavior issues, not just syntax? Compliance gaps, silent failures on edge cases, non-deterministic outputs at runtime?

We're building toward deterministic verification at FetchSandbox specifically because of this pattern. But I want to understand how teams are handling it today before we get too deep into assumptions.

what's actually working for you, or what's broken. Both are useful..looking for some feedback ..

if u are building and deploying agents in prod which takes actions on behalf on consumers … how are u validating agent actions and their execution runtime?

me and my friend who quite meta last Dec and building agent verification engine, with ai building is cheap now but validading weather agent does right job or not determinstically still a gap we are seeing.

we strated around april/may and launched on Ph two times and ranked top 3,2 ..

and spoke to developers and builders who are deploying agents on prod.

i wanna see feedback from community how are you all validating weather or not agent is taking right actions and if it’s not doing its job how are u foxing it runtime so that it stops the bleed.

Spent the last few weeks actually talking to founders. Cold DMs, intros, random convos. Never done sales before, always been an eng. It's humbling in a good way. You show up, you listen, you learn fast.

One thing keeps coming up: agents that take real actions have no clean verification story. Talked to a founder building an agent that posts and interacts on X. To check if it worked, they were hitting the real API. Real calls, real money, real risk of getting rate-limited or banned mid-test. Agent fired 40+ API calls in one test run, X flagged the account, $30 gone, no useful signal.

Build is fast now. Verification is still stuck at "run it in prod and hope."

For people building agents that take real actions: what does your verification loop actually look like today?

Spent 3 days debugging a Paddle billing issue that I couldn't reproduce locally. Turned out subscription.activated was arriving before subscription.created in prod about 30% of the time. My Lovable-generated webhook handler looked clean, passed every test, handled retries. But it assumed events arrive in order. They don't.

The local sandbox never showed this. Customer's app was stuck on "pending" while Paddle showed "active." No error, no stack trace, just vibes.

Ended up building a stateful Paddle twin to inject out-of-order delivery and retry storms just to catch this stuff before it hits prod. Found two more race conditions in the first week.

Anyone else running into this with webhook handlers built on Lovable? How are you actually testing these before shipping?

Grateful for the kind words from this community last week

Saw a few encouraging comments on my previous on VC demo/grinding 3am..early hours...( this post ) and it genuinely helped.

Had a call today at 2pm, scheduled for 20 mins, ended up running 43. Me and my co-founder walked away feeling really good about where things are headed.

Last 2.5 months we were heads down on distribution after we validated that the deterministic verification engine actually works in real production scenarios.. agents running FetchSandbox MCP to verify integrations before they hit prod, and the receipts showing up in PRs. That's the thing that clicked for people on the call too.

The idea came from pain at my last job honestly.. never thought it'd take this shape. Next big thing we're hardening is user auth. Distribution-first worked, now it's time to tighten the core.

Will post an update once we have the next steps locked.

Thanks again for the support here, it means more than you think when you're building solo weekends.

genuinely curious what the flow looks like for most people here.

you build something with stripe or twilio or resend wired up, it looks right in the builder, maybe you click through it once... then you ship. and then prod surfaces the weird edge case. flaky webhook, card decline mid-flow, double charge on retry.

is there a verification step most people run before launch, or is it mostly ship and find out?

Genuinely asking because i don't see this talked about much here.

You prompt your way through a stripe or twilio or resend integration, it looks right, maybe you test the happy path once... then you ship. prod surfaces the thing you didn't test. flaky webhook, card decline mid-flow, retry fires twice and double-charges someone.

is there a verification step most people are running, or is it mostly ship and find out?

genuinely curious what the flow looks like for most people here.

you build something with stripe or twilio or resend wired up, it looks right in the builder, maybe you click through it once... then you ship. and then prod surfaces the weird edge case. flaky webhook, card decline mid-flow, double charge on retry.

is there a verification step most people run before launch, or is it mostly ship and find out?

Genuine question. I keep seeing teams ship MCP integrations and then find out something broke because the agent made an unexpected API call, the webhook payload came in slightly differently, or a partner quietly changed their schema and nobody caught it.

Do you manually trace what the agent did? Read logs after the fact? Hope staging catches it?

The drift problem is what gets me most. Your MCP tool works fine today, your partner pushes a format change next week, and your agent just starts failing silently or doing the wrong thing with no obvious signal.

We're building a sandbox that runs your MCP tools in a stateful environment so you can verify what the agent actually executed, before you merge, not after prod breaks. Still early but already using it ourselves daily.

how others are thinking about this. Is verification even on your radar yet or are most teams still just shipping and watching?