Client burned $900 in one day chasing 'AI-first' instead of doing the boring thing that would've worked
A client of mine runs a business with a few different portals and websites. Every time his dev team ships an update, someone has to manually go through the apps to make sure nothing broke. He wanted that manual checking automated.
I said the sensible path was to have his developers write automated checks inside the codebase itself. Standard practice. Every serious software project does this. Cheap. Fast. Reliable. It's not fancy, but it works, and it costs basically nothing to run.
He didn't want that. He wanted AI. AI was going to solve everything, apparently.
So we built the AI version. Months of work. It uses AI to look at each screen of his app the way a human would, and decides whether things look right. He ran it on one feature the day we handed it over.
$900 API bill by end of day.
His Slack message was one word: 'Bro.' Attached the invoice screenshot. Holy hell.
Now the awkward part: the tool actually works. Really well. It does exactly what a human tester would do. Catches things a person might miss. Documents everything with screenshots. Genuinely impressive.
But $900 per feature-check isn't testing. That's a mortgage payment.
If we'd done the sensible thing from day one — normal automated tests inside the code, and only used AI-vision for the parts normal tools genuinely can't handle (things like Google Maps autocomplete fields, custom date pickers, weird embedded widgets) — the same run would've cost him around $30. And it would've been 10x faster.
The lesson I keep coming back to: AI is a scalpel, not a sledgehammer. Slapping it on every step of a process because 'AI is the future' is how you turn a $30 job into a $900 one.
This is happening everywhere right now. Business owners hear 'AI' and want it in every corner of the company. Sometimes it's the right call. A lot of the time it isn't. The hard part is figuring out which BEFORE you get the $900 bill.
Anyone else seen this? Where has AI actually paid off in your business, and where did it turn out to cost more than the problem was worth?