u/No-Law-5350

Every mobile team we've watched write "acceptance criteria" ends up with two categories: the criteria the PM wrote, and the criteria the engineer added because the PM's list was full of gaps. The PM's version is usually "user should be able to X." The engineer's version is "what happens when the network drops mid-X, when the user has no items yet, when the token has expired but not yet refreshed."

Most PMs just don't think in states and transitions. Engineers do because their whole job is caring about states and transitions. When PMs write test cases, you get happy path bingo cards.

If your acceptance criteria are still 80% "user should be able to," your test plan is going to be, too.

Test driven development thrived on backend Java in 2004 because functions were pure, dependencies were injectable, and unit boundaries lined up with test boundaries. Mobile doesn't work that way. Your UI state is entangled with lifecycle, your platform APIs are stubbable but not really testable, and the interesting bugs happen at the intersection of components you can't unit test.

Every mobile team that adopted TDD in 2015-2018 either quietly dropped it or watched their coverage climb while production bugs stayed identical. The 2020s consensus in mobile was to invest in integration and end-to-end tests instead. Nobody wrote a post-mortem on TDD-for-mobile because it wasn't a decision. It was a slow abandonment.

If you're still writing red-green-refactor for View state, you're the last one at the party.

most have never watched a good tester use an app. A senior manual QA will find a broken flow in 15 minutes by wandering into corners no automated test would ever visit, background the app mid-payment, rotate three times, spam the retry button, close the network mid-request, and resume the app three days later.

Every "we automated everything" team we've talked to eventually rehires manual QA. Usually 8-14 months in. The framing changes to "exploratory testing" or "bug bash," but it's the same job. The bugs those testers find are the ones that make it to production reviews.

If your automation strategy assumed you'd get rid of manual testers, revisit that assumption. The value isn't in the ratio. It's in the coverage overlap.

Most mobile CI pipelines do two things: confirm the app builds and run the full UI suite against every commit. The first is necessary. The second is almost always wrong. You don't need to run your payment flow test on a PR that only touched the settings screen.

Affected tests only run (compute affected tests from git diff, file path, or dependency graph) and cut most teams' CI time by 60-80%. The architecture takes a weekend to set up with Nx, Bazel, or Turborepo. The reason almost nobody does it is that no one on the team has the headspace to challenge the existing pipeline.

Our take: "CI is slow" is almost never a tools problem. It's an unchallenged assumption problem for some reason.

We pulled the data from customer test runs. Teams running visual regression (pixel diff or semantic diff between releases) catch about 30% of UI bugs that no other test layer catches. Mostly: button cut off by safe area, text overflow on smaller screens, contrast regressions, alignment shifts after a refactor that "didn't change the UI."

Despite that hit rate, fewer than 1 in 5 mobile teams we've talked to runs any visual regression. The reason given is always "too noisy." True if you do whole screen pixel diff. Not true if you compare semantic regions with diff tolerance. Three lines of config in any modern tool.

Our take: visual regression is the highest ROI testing investment most mobile teams aren't making.

Shift left tells mobile teams to push testing earlier in the dev cycle. Sounds great. The actual implementation in most orgs is dev engineers writing brittle UI tests that then rot in CI because the same devs have no time to maintain them. Coverage looks higher. Real bug detection drops.

Mobile is different from backend. The UI churns too often, the testing primitives are heavier, and the cost of ownership for automation is real. The right move isn't to shift testing left. It's to shift the right work to the right level. Smoke + visual tests on every PR. Heavy E2E on nightly. Exploratory + perf on release.

Our take: most "shift-left" mobile initiatives produce worse outcomes than the model they replaced.

Recorded tests have one specific failure mode. They pass once during recording, then break on the first UI change because the captured selectors are coordinates or DOM paths that don't survive a refactor. Teams treat the test as "working" until the day it breaks, then call it flaky.

The honest framing is the recorded test was never working. It was a coincidence that the captured selectors happened to match on day one. The first failure isn't regression. It's the test's first encounter with reality.

Our take: if you can't rewrite the test in plain language without recording it, you don't have a test. You have a fragile demo.

We've talked to a few hundred QA engineers in the last year. The pattern is consistent. They spend 60-70% of their time maintaining brittle automated tests that mostly catch their own framework's drift, not product bugs. The remaining 30% is exploratory testing they're good at, that their org doesn't respect.

Every QA lead we've talked to wants to invert that ratio. Their managers want more automation coverage. The result is like a treadmill where QA writes more tests to prove they're "automating" while bug-finding work gets squeezed.

Our take: the metric that matters is "bugs found per QA hour," not "tests automated per quarter. " Most orgs measure the wrong thing on purpose.

We hear "we accept 10% flake rate as the cost of automation" constantly. That's a convenient way to ship without admitting your tests don't work. Every accepted flake is either a real bug being hidden by retries or an unreliable test training your team to ignore failures.

The honest move is to demote flaky tests to "advisory" status that doesn't fail the build, then fix or delete them on a schedule. Splitting the difference (flaky tests still gate, but you accept the noise) rots the suite over months.

Our take: most teams we've seen with a "flake budget" is on the slow path to abandoning automation entirely. The flake budget is the leading indicator.

A 90% coverage suite where most of those lines are happy-path getters is strictly worse than a 40% coverage suite that hits actual failure modes. We've closely watched teams ship features behind 95% coverage that broke immediately in production. And teams with 35% coverage that handled launches without incident.

The signal isn't how many lines your tests touch. It's which lines, what assertions, and what production bug each test would catch. If you can't answer "what real bug would this test prevent?" the test is theater.

So What we say is track "bugs the suite caught last quarter" instead of coverage. The number will be uncomfortably low. That's the point.

We've been writing Page Object Model code since 2013. The original reason was specific: selectors broke constantly, so we abstracted them into one place to limit the blast radius. The pattern stuck because there was no better option.

That problem is largely gone now. If a test framework finds elements visually instead of by selector, you don't need an abstraction layer between "tap login button" and the actual login button. The page object exists to absorb churn that no longer exists in the underlying framework.

Our take: writing page objects for new tests in 2026 is paying tax on a problem that's been solved.

iOS 17 broke WebDriverAgent in ways that still aren't fully fixed in 2026. The Appium repo PR velocity has been declining for 18 months. Every major mobile testing talk at droidcon and KotlinConf last year was about something else: Maestro, vision AI, and Compose-native frameworks. The community has moved.

Our take: if your team is still on Appium because of inertia, that's a choice. If it's because you genuinely evaluated Maestro, Detox, and other tools and Appium won, that's fair. But "we use Appium" is no longer a safe default in 2026. It's a position you have to defend.

adb shell pm grant <package> <permission> grants permissions to your Android app before the test starts. Drop it in your test setup. You skip 4-8 seconds per run tapping through location, notification, or microphone dialogs, and your tests stop breaking when Android changes the permission UI between major versions. This works on real devices and emulators.

For iOS simulators, xcrun simctl privacy <simulator> grant <service> <bundle-id> does the same thing. Pre-grant photos, location, contacts, or microphone before the test starts and the dialog never appears. Note that simctl privacy is simulator-only. On real iOS devices, you still need the dialog or a UI automation tap.

If your test was passing two weeks ago and is now failing 30% of the time, you can let git bisect find the responsible commit automatically. Write a shell script that runs your test 10 times and exits 1 if more than 2 fail. Pass it to git bisect run ./flake check.sh. Git checks out commits between your known-good and known-bad SHAs, runs your script on each, and points to the commit that introduced the flake.

We've used this to trace flakes back to "harmless" upgrades  a Compose version bump, a coroutines patch  that quietly changed timing assumptions nobody noticed in code review.

If your React Native app opted out of Hermes (the JavaScript engine bundled with RN), your migration window is closing fast. RN 0.81 removed JavaScriptCore from core. RN 0.84 made Hermes V1 the default. JSC still works through @/react-native-community/javascriptcore on RN 0.79+ with the New Architecture, but the package has no config plugin yet, so you wire it into the native iOS and Android projects by hand.

The teams affected are the ones that had a specific reason to stay on JSC. Maybe a native module that depended on JSC specific behavior. Maybe a bundle size optimization. Maybe a debug workflow that needed Safari DevTools. You can keep JSC through the community package, but it's now a maintenance cost you own, not something Meta ships for you. For most teams, migrating to Hermes is the cheaper long term call.

Migrating is usually smooth, but three edge cases are worth auditing first. Regex engine differences (Hermes uses its own RegExp engine, not JSC's, and a few patterns behave differently). BigInt handling. Some Intl APIs that Hermes implements differently from JSC. If your app touches any of those three heavily, plan more migration time. If it doesn't, the switch is usually a build config change and a smoke test pass.

Android 15 (API 35) enforced edge to edge by default. Apps that hadn't migrated saw Material 2 top app bars, FABs, and list items obscured by the status bar, the 3-button navigation bar, or the display cutout, depending on the device. That was the soft enforcement.

Android 16 (API 36) removed the escape hatch entirely. The R.attr#windowOptOutEdgeToEdgeEnforcement flag, which teams used to delay migration, is deprecated and disabled in API 36. There is no opt out for apps targeting 36. Apps targeting 36 still honor the flag on Android 15 devices, but the moment that device updates to Android 16, the flag is ignored.

For UI test suites, this means a wave of false positive failures as elements move under system bars. Tests that asserted on top bar visibility started failing on Android 15+ devices, and Android 16 made those failures permanent.

Material 3 Scaffold or explicit WindowInsets handling are the migration paths. If your test suite runs on emulators upgrading to API 35/36, expect a regression spike before you've changed any code. The platform changed under you.

React Native 0.85 shipped in April 2026 and removed the Bridge from the codebase entirely. RN 0.82 in late 2025 already made the New Architecture non optional, but 0.85 is where things actually break in production for teams that hadn't fully migrated.

Four breakages we keep seeing in customer apps. setNativeProps no longer works, refactor to state or Reanimated. Some async layout measurement patterns run synchronously now, which changes timing assumptions in animations. Gesture timing changed for custom gesture logic. Apps that opted out of Hermes don't run on the New Architecture at all because JSI requires Hermes.

The other surprise is the interop layer that existed between 0.76 and 0.84 got removed in 0.85. Library maintainers who relied on it ship libraries that compile fine and crash at runtime. Audit every native dependency before upgrading. Most migrations take 2-8 weeks, and the third party library audit is the variable.

If you're running Appium on iOS 17 with Xcode 15, this one is documented in the wild and probably explains a lot of your CI slowness. Multiple open GitHub issues on the appium/appium repo report that fetching the page source via WebDriverAgent went from a few seconds on iOS 16 to over 4 minutes on iOS 17, with Xcode 15. Several teams have flagged WebDriverAgent build failures and runtime crashes after extended runs as well.

These aren't edge cases that affect three users. They're long running issues with active comment threads and partial workarounds. The Appium team is working through them, but the regression on iOS 17 is real, and the "fix" for most teams is either downgrading their Xcode toolchain or accepting longer test runs.

Vision based testing doesn't depend on WebDriverAgent at all, which is one of the reasons we went down that path. The dependency on Apple's internal accessibility tree is a tax that gets bigger every iOS major.

12 mobile teams that previously used Appium agreed to log their test maintenance time for 6 months while we ran the same tests in parallel with Drizz.

Median Appium maintenance was 4.7 hours per app per month. About 1.8 hours on locator updates, 1.1 on wait timing, the rest on driver config and rewrites after UI changes. Drizz median was 0.2 hours. That's a 23x reduction.

The interesting part is what we couldn't measure. Across all 12 teams, we counted 47 test files marked as "skip" in CI for more than 30 days. Those tests were being done manually by QA every release. Estimates of manual hours those skipped tests cost ranged from 8 to 32 per team per month.