r/AITestingtooldrizz

most have never watched a good tester use an app. A senior manual QA will find a broken flow in 15 minutes by wandering into corners no automated test would ever visit, background the app mid-payment, rotate three times, spam the retry button, close the network mid-request, and resume the app three days later.

Every "we automated everything" team we've talked to eventually rehires manual QA. Usually 8-14 months in. The framing changes to "exploratory testing" or "bug bash," but it's the same job. The bugs those testers find are the ones that make it to production reviews.

If your automation strategy assumed you'd get rid of manual testers, revisit that assumption. The value isn't in the ratio. It's in the coverage overlap.

A 90% coverage suite where most of those lines are happy-path getters is strictly worse than a 40% coverage suite that hits actual failure modes. We've closely watched teams ship features behind 95% coverage that broke immediately in production. And teams with 35% coverage that handled launches without incident.

The signal isn't how many lines your tests touch. It's which lines, what assertions, and what production bug each test would catch. If you can't answer "what real bug would this test prevent?" the test is theater.

So What we say is track "bugs the suite caught last quarter" instead of coverage. The number will be uncomfortably low. That's the point.

We've talked to a few hundred QA engineers in the last year. The pattern is consistent. They spend 60-70% of their time maintaining brittle automated tests that mostly catch their own framework's drift, not product bugs. The remaining 30% is exploratory testing they're good at, that their org doesn't respect.

Every QA lead we've talked to wants to invert that ratio. Their managers want more automation coverage. The result is like a treadmill where QA writes more tests to prove they're "automating" while bug-finding work gets squeezed.

Our take: the metric that matters is "bugs found per QA hour," not "tests automated per quarter. " Most orgs measure the wrong thing on purpose.

We hear "we accept 10% flake rate as the cost of automation" constantly. That's a convenient way to ship without admitting your tests don't work. Every accepted flake is either a real bug being hidden by retries or an unreliable test training your team to ignore failures.

The honest move is to demote flaky tests to "advisory" status that doesn't fail the build, then fix or delete them on a schedule. Splitting the difference (flaky tests still gate, but you accept the noise) rots the suite over months.

Our take: most teams we've seen with a "flake budget" is on the slow path to abandoning automation entirely. The flake budget is the leading indicator.

I was one of 40 engineers laid off in a round that was framed as "right-sizing." I'd been at the company for six years. I was the only person who fully understood our legacy reporting system, which was a Frankenstein of stored procedures, cron jobs, and a Python service I'd written in my first year.

The layoff itself was clean. Severance was reasonable. I went home Friday afternoon and started updating my resume.

On Monday morning I got an email from someone in finance asking if I was "open to consulting on a critical migration project for a few months." It turned out a quarterly report had broken over the weekend and nobody on the remaining team could fix it. They had a board meeting on Thursday. They needed the report.

I quoted three times my old hourly rate, billed in monthly retainers with a two month minimum. They agreed within an hour. I'm now in month seven of what was supposed to be a "few months." I've made more in consulting fees than my full year salary plus severance.

The same VP who approved my layoff signs my invoices. We've never discussed it.

I got a $25 Starbucks gift card.

I was debugging a customer complaint about a duplicate charge. While tracing the code path, I noticed our retry logic was triggering on a specific timeout case where the payment had actually succeeded. We were re-charging customers when the original charge had cleared but our system hadn't gotten the confirmation in time.

I pulled the logs. It had been happening for about 14 months at a rate of roughly 200 charges per month. We'd been quietly issuing refunds when customers complained, but most of them never noticed because the duplicate posted on their statement as a separate line item.

The financial impact was somewhere around $480K in incorrect charges, plus the refund processing fees, plus the chargeback fees from the customers who escalated to their banks.

The fix was changing one boolean check and adding an idempotency key. About 10 minutes of work and a 6-line PR.

The company's response was a recognition post in the engineering Slack channel and a $25 gift card from the "shout out" program. The same program gives out gift cards for people who organize the team lunch. I left three months later.

iOS 17 broke WebDriverAgent in ways that still aren't fully fixed in 2026. The Appium repo PR velocity has been declining for 18 months. Every major mobile testing talk at droidcon and KotlinConf last year was about something else: Maestro, vision AI, and Compose-native frameworks. The community has moved.

Our take: if your team is still on Appium because of inertia, that's a choice. If it's because you genuinely evaluated Maestro, Detox, and other tools and Appium won, that's fair. But "we use Appium" is no longer a safe default in 2026. It's a position you have to defend.

at $40K more than I was asking. I trained him.

I'd been a mid level engineer for four years at same company. Each promotion cycle my manager said I "wasn't quite ready" and gave me a stretch project to prove myself. I delivered each one. and got same feedback.

After the third rejection, they posted a senior role externally. The job description was almost word for word what I'd been doing for two years. I applied to it internally as a joke. HR told me I "didn't meet the qualifications."

They hired someone from outside a month later. His start date came through in our team Slack channel. His starting salary was leaked in a comp doc that got accidentally shared the next quarter. He was making $40K more than the number I'd asked for in my last review.

My manager asked me to "onboard him to the codebase." I did it. It took six weeks. He's a fine engineer. He's also six years younger than me with two fewer years of experience.

I left four months after he started. He still messages me on LinkedIn occasionally to ask questions about the system.

If your test was passing two weeks ago and is now failing 30% of the time, you can let git bisect find the responsible commit automatically. Write a shell script that runs your test 10 times and exits 1 if more than 2 fail. Pass it to git bisect run ./flake check.sh. Git checks out commits between your known-good and known-bad SHAs, runs your script on each, and points to the commit that introduced the flake.

We've used this to trace flakes back to "harmless" upgrades  a Compose version bump, a coroutines patch  that quietly changed timing assumptions nobody noticed in code review.

I know this is going to upset people but I've seen too much damage to stay quiet about it

we hired a mid-level engineer last year, strong React background, could build anything on the frontend, knew his ORM inside out, got the job done consistently, everyone liked him, and then we gave him a small database migration task and he caused a 4-hour production outage because he had genuinely no idea what was happening underneath the abstraction he'd been using for 3 years

not his fault entirely, nobody ever made him learn it, the tooling made it possible to be productive without that knowledge so he never built it, but the result was a team of users who couldn't access their data for 4 hours and a very uncomfortable postmortem

and the thing that scares me is this is not rare anymore, it's becoming the norm, I've spoken to engineering managers at 6 different companies in the last year and every single one of them described some version of this story, smart capable developers who are completely lost the moment the abstraction breaks

ORMs are not the problem, abstractions are not the problem, the problem is treating them as a replacement for understanding rather than a tool built on top of understanding

you should know what your ORM is doing before you get to use it unsupervised in production, full stop, this is not a gatekeeping take it's a damage prevention take and I'm tired of pretending otherwise

every senior engineer I've told this to has had the same reaction, a sharp inhale followed by "why would you do that," and I get it, the conventional wisdom is that database access is earned slowly over years and guarded carefully by the people who understand it best

but here's what actually happened, the junior dev had good tooling, she could see the schema clearly, understand the relationships, trace what a change would actually do before doing it, and because of that she moved faster and made better decisions than developers twice her experience who were working blind with a vague mental model and a prayer

we treat database access as dangerous and therefore restrict it, but I think we have the causality backwards, it's not dangerous because of access, it's dangerous because of ignorance, and the solution to ignorance is visibility not gatekeeping

the engineers who break production databases are almost never the ones who understand them too well, give people the tools to actually see what they're working with and most of them will surprise you

I joined as a mid level engineer and during my first architecture review, I noticed the caching layer was writing user profile updates to a local cache and syncing to the database asynchronously. If two requests hit different app servers within the sync window, the second write would overwrite the first. 

I raised it in the review and the staff engineer who designed the system said "the sync window is 50 milliseconds, the probability of two profile updates from the same user hitting different servers in 50ms is negligible." I said "it's not negligible if you have a form that auto saves on every field change” and our profile page had 11 fields. A user tabbing through the form and editing 3 fields would fire 3 PUT requests within 200m and with round robin load balancing, they'd likely hit different servers.

He said I didn't understand how the load balancer worked and I seriously dropped it. I was new but he had 14 years of experience and I assumed I was missing something.

Six months later (last week) support got a cluster of tickets. Users said their profile changes were "reverting." A user would update their phone number and address, come back the next day and the phone number was correct but the address was old or the other way around sometimes. The staff engineer debugged it for 2 weeks and the root cause was exactly what I'd described haha. 

He fixed it with a database level optimistic lock but he never mentioned my original comment and I didn't bring it up either.

I spent 6 months building an internal tool that automated our data pipeline monitoring and before the tool 4 engineers spent roughly half their time manually checking pipeline health, investigating alerts, restarting failed jobs, and writing status reports and my tool automated all of it. I demoed it to leadership and got a standing ovation (figuratively, it was google meet). My director called it "the most impactful internal project this quarter" and the VP of engineering mentioned it in all hands plus I got a spot bonus.

Two weeks later, a reorg was announced and my own team of 6 was reduced to 3. The three who were let go were the ones whose daily work had been most automated by my tool and the reorg doc cited "operational efficiency gains" as the justification. Nobody blamed me directly but the math was obvious. I built the thing that proved their jobs could be done by software but I wanted to free my teammates from tedious work so they could focus on harder problems and leadership wanted to free themselves from my teammates' salaries.

I've built internal tools since then and I'm more careful now about how I frame the value because I learnt that efficiency in the wrong context isn't a gift to your team. That and a couple tools are still running and the team is still 3 people. They handle the same workload that 6 people did but nobody got a raise.