I built my software 100% vibe-coded, and now I need to tighten up QA.

I’ve already set up around 200 automated Playwright tests for the software. I created the test cases using screenshots from both the mobile app and desktop version. I’m also using Check now to test workflows like invoicing, since that’s one of the features inside the app.
I’m curious what other vibe coders, or people working with mostly AI-generated codebases, are doing to reduce bugs and errors. Right now I have a staging environment with a new version of the software that I’m also unit testing each feature on using Codex as that is my primary AI coding tool. I’ve used Claude code, but I really prefer Codex and I have some technical background. I completed a degree in computer science southern private school so I’m not clueless when it comes to the software development life cycle but for vibecoding specifically I’m curious what protocols people have used to cut down on production bugs. To me it seems like with AI you can build much faster, but the testing process needs to be much more thorough than it needed to be before. I catch them by looking at post hog, and sometimes the users tell me directly, but the ones who onboard itself self serve without me talking to them thought the software was broken. Partly because to do the invoicing feature you needed to connect your Stripe account or Square account and I was doing that part of the onboarding on the phone and I’ve now moved up doing this step in the onboarding process but steal, I think that just goes to show that I really need to cut down on bugs. I get regression with sign in as well so I’ll fix a bug with sign in and then two days later. People can’t login with Google again. It’s extremely aggravating.

Right now, I dogfood the product, do manual testing, and run automated Playwright tests every morning around 4 AM. Sometimes the tests give me useful insight, and sometimes I question their reliability, but overall I know I need much stronger QA to reduce production bugs.
The hard part is that the software is already pretty extensive. I have around 70–100 features, so there are a lot of edge cases. Even with dogfooding and manual testing, users still report bugs I didn’t catch.

Marketing is working pretty well through Meta ads, and people are ready to use the software. The problem is that a lot of trials don’t convert because users hit bugs. The UI is also confusing, but I’m not focused on fixing that yet. The bigger issue right now is literal software bugs breaking the experience.

I built my software 100% vibe-coded, and now I need to tighten up QA.

I’ve already set up around 200 automated Playwright tests for the software. I created the test cases using screenshots from both the mobile app and desktop version. I’m also using Checkly now to test workflows like invoicing, since that’s one of the features inside the app.
I’m curious what other vibe coders, or people working with mostly AI-generated codebases, are doing to reduce bugs and errors. Right now I have a staging environment with a new version of the software that I’m also unit testing each feature on using Codex as that is my primary AI coding tool. I’ve used Claude code, but I really prefer Codex and I have some technical background. I completed a degree in computer science southern private school so I’m not clueless when it comes to the software development life cycle but for vibecoding specifically I’m curious what protocols people have used to cut down on production bugs. To me it seems like with AI you can build much faster, but the testing process needs to be much more thorough than it needed to be before. I catch them by looking at post hog, and sometimes the users tell me directly, but the ones who onboard itself self serve without me talking to them thought the software was broken. Partly because to do the invoicing feature you needed to connect your Stripe account or Square account and I was doing that part of the onboarding on the phone and I’ve now moved up doing this step in the onboarding process but steal, I think that just goes to show that I really need to cut down on bugs. I get regression with sign in as well so I’ll fix a bug with sign in and then two days later. People can’t login with Google again. It’s extremely aggravating.

Right now, I dogfood the product, do manual testing, and run automated Playwright tests every morning around 4 AM. Sometimes the tests give me useful insight, and sometimes I question their reliability, but overall I know I need much stronger QA to reduce production bugs.
The hard part is that the software is already pretty extensive. I have around 70–100 features, so there are a lot of edge cases. Even with dogfooding and manual testing, users still report bugs I didn’t catch.

Marketing is working pretty well through Meta ads, and people are ready to use the software. The problem is that a lot of trials don’t convert because users hit bugs. The UI is also confusing, but I’m not focused on fixing that yet. The bigger issue right now is literal software bugs breaking the experience.

Check out my prior posts: SaaS Churn Analysis: $100 MRR June 3rd - July 3rd

1. Low Business Activity: They came in because they didn’t have a way to invoice a $90-per-quarter customer, and they generated only a couple of other leads. Marketing needs to target businesses with more established activity. Even businesses with real activity but stuck in pen-and-paper inertia may not be the best segment, but they are less likely to churn simply because the subscription costs more than their revenue.  
2. Might have seen AI slop comments since I know I get a lot of return users and reactivation (not a bad PMF signal)
3. High-touch onboarding: My onboarding was building the features for him, making it extremely high-touch, but it might have been entertaining for him to see it built that quickly. Entertainment is not the highest retention.
4. Repeated product bugs: The most extreme was before he bought the subscription, and I had started the mobile app. The mobile app would show up on the laptop. This codebase has been moved across four different tools and one database migration. This is also a massive codebase compared to anything I've built before. 
5. Rushed sale: I thought this was a better way to sell, but he tried to push it to June 10th before he bought the software because he was concerned about bugs. I still tried to get it on the third to not lose the sale, and I told him it was likely not possible. I think the sales won the deal more than the product did.
6. He didn’t want to schedule time to set up the AI receptionist I tried to build before the trip. Going forward, that could be treated as a churn signal, a high-risk account indicator, or a sign that the customer needs more attention. That said, I don’t think this one moment caused the account to go bad. Structurally, it is hard to retain customers when the software still has too many bugs. What I gained in speed, I sacrificed in quality. Speed is still necessary, but quality needs to be more automated and treated as a bigger operational priority going forward.
7. I accepted feature requests for systems I couldn't properly QA because I didn't have access to them. I should stop doing that. WordPress was one example: I couldn't test it without the customer. A glove integration is another.
8. New operators are high risk if there isn't enough business activity, even if there is appeal on the acquisition time since these won't be rip-and-replace deals.
9. Businesses with 5 leads a week might be a better segment as long as they don't have software.
10. He seemed the most tech-savvy among everyone I have spoken with.
11. New operators are better than existing operators with no software if there isn't inertia.
12. If churn will be this high, I need more revenue opportunities. The solution is better marketing, or a bigger TAM is needed. If churn is going to be this high, I need more revenue opportunities. The answer is either better marketing or a bigger TAM. I already made 300 Instagram scrolling posts that I will schedule soon, and I am switching ad tools and comment blocker to reduce AI slop comments.
13. Some of these businesses just need better marketing to survive.
14. Was willing to vouch in a Pest Control Facebook group and still churned.
15. The biggest issue is chasing new customers too quickly after closing one. I built for new prospects when I should've stayed closer to the customer I already had. The software still needed time to mature, so more pipeline also meant more strain. I had a daily text cadence, but after the trip he became harder to reach, and the high-risk WordPress integration created frustration.
16. The "new business as first market" thesis may be flawed, not on acquisition, but on retention.
17. Lower-tech operators may be better customers. They're less likely to game trials or churn quickly.
18. Marketing shouldn't drift too far into new hooks. The old hooks should keep being repeated because he still matched the ICP in several ways.
19. Facebook requires high touch.
20. He was made aware of the 50 new features that are coming this week. 
21. AI development meant days I could iterate on his feedback 3x, but it created massive tech debt with the stack I used (15,000 lines of mobile code before refactor).
22. Self-serve users thought the product was broken, but they legitimately tried to use it. Now I'm forcing them to set up Stripe without talking to me first. Before, I collected their phone numbers, cold-called them to onboard, and followed up by SMS.
23. He was not self-serve, but the mobile product was literally built with him. He was the one that gave me all the screenshots for incumbent 2nd biggest in market competitor.
24. Software bugs were extreme when he first got on the desktop platform as we built the mobile app in parallel.
25. It began to seem like he was patient and would not need to be as high-touch either.I think his business stalled when he wasn't able to hire a salesperson on Indeed.  
26. He still has an annual 11-month contract on the app. When I speak on the phone with him, I will notify.
27. I am unsure why he would cancel when he came to me before purchasing the incumbent 2nd biggest in market competitor and mentioned the audit angle it could solve. I bet he swicthed to the best player in the market

Faster markets create more revenue opportunities (set up calls). I'm getting high-quality leads, but not enough of them yet.

I am testing 50-60 new features to match market winner competitor since they compare me to them on Facebook Ads, but I need to beat them on production quality and bug control. To move up the depth chart, I have to be 2x better, then the players ahead. I built the 50-60 before I knew he was going to churn. 

I can't feel entitled to keep a customer's business. Facebook leads require high-touch onboarding. Before the UI upgrade, my second product in a different market was 100% self-serve and card-gated, but very few users entered a card, and those that did eventually churned. If acquisition is high-touch, retention will likely need to remain high-touch as well, limiting scalability and increasing servicing costs. My second product in a different market had an activation lift after self-serve was not working in the model.

The most motivated customers will find a way to book time on the calendar, even if it isn't prominently displayed my second product's activation improved once I realized a purely self-serve onboarding flow wasn't viable given the product's maturity.

The email campaign should continue, but automated testing and QA need even greater emphasis to support a scalable onboarding model.

Moving as fast as I did was impressive and helped draw him in, but it also set the wrong expectation for how quickly his requested features would be built while I was also talking to other customers.

The timelines were still fast. Even the AI receptionist may have only taken two days, but the WordPress hiccups turned something initially shipped in three days into a 10-day process to get close to the correct version, mainly because I could not properly QA it myself, and I could only get him on the laptop once a day.

I think it's important to remember this is just one person. Sometimes it's a little bit opaque on whether you need to make huge business changes or if it's better to put through. The truth is likely in the middle. The smartest option would be likely to try to keep up my current lead volume, but for a higher ticket segment that can fit the ideal customer profile of not having a whole lot of pen and paper inertia, but maybe a weak competitor like weakest incumbent, or just already having success from fast growth  and I identify the ones hurting the most badly in this situation  

This churn might not be final, but it is still worth analyzing.

• The card failed, and it looks like the customer may be done. I tried to move fast on the AI receptionist and even attempted same-day delivery, but this time I wanted to build it with better QA instead of rushing it out.
• The concerning part is that he asked me to build the feature, but then was not willing to set it up. That matters because the AI receptionist should have been a strong retention feature. I tried onboarding him over text, but he did not complete the setup.
• I thought I had more time because he was also asking for advice on recruiting and marketing, but his responses started slowing down even though I kept a daily cadence.
• I'm not going to cope. The software still needs work. But this is surfacing useful concerns early. The demand is there. It's my software that's killing me. I am my own worst enemy right now.
• The bigger question is whether $100/month is more expensive than I thought if each customer requires this much white-glove onboarding and high-touch retention. Retaining 60 customers at this level of service could become difficult. I might need higher prices.

All I had to do was think of a niche industry that had customers with recurring customers so I could have good retention. I put out a webpage with a $1 paywall to access a Lovable basic MVP for this one and 30 others. After that, this page somehow got indexed on Google Search and found a couple of people who were willing to spend $1 without talking to me.

At that point, I built up the full product and ran small Meta ads, like $10 a day and no more than $20. I saw that I was getting a good number of leads, and every time they came in, I called as soon as possible and sent a text message.

I met one customer who wanted one of the products, and I built a lot of custom features that matched the competitors while also squashing bugs along the way. I found that sharing videos of the software and showing mockups before writing any code were good for software activation, iterating, and building the correct thing. He wanted a mobile app, so CapacitorJS was also fantastic for quickly turning PWAs into mobile apps. I still have bugs but I managed to get the app into the App Store and completely finished in under 3 weeks.

This is vertical software, so there are many features, but I was so happy that he paid today after years of grinding. My next goal is five paying customers, but going forward I want to sell the product without building as many custom features so I avoid product bloat down the line. Retention will be everything!

I think I might’ve cracked maximum human productivity with my two laptops. I’m able to manage anywhere from 10 to 12 agents using Claude Code for most waking hours and at least 2 as I sleep as well that pair with my 155-244 words per minute typing speed and hatred for all sunlight productivity. I typically split between coding tasks and browser automation. Side effects are my laptop, constantly crashing from too many tabs but I have now automated the terminal commands for that as well with AI. I have now ran red teams against my own software as an ethical hacker as well to bulletproof my software. Along with that, I’ve ran quality assurance over a dozen times to be sure all the buttons work as I have turned that process into a cron job that runs every morning.

I think I might’ve cracked maximum human productivity with my two laptops. I’m able to manage anywhere from 10 to 12 agents using Claude Code for most waking hours and at least 2 as I sleep as well that pair with my 155-244 words per minute typing speed and hatred for all sunlight productivity. I typically split between coding tasks and browser automation. Side effects are my laptop, constantly crashing from too many tabs but I have now automated the terminal commands for that as well with AI. I have now ran red teams against my own software as an ethical hacker as well to bulletproof my software. Along with that, I’ve ran quality assurance over a dozen times to be sure all the buttons work as I have turned that process into a cron job that runs every morning.