AI is overrated
I joined a company a while ago - it's a UK company that was acquired by big US tech. The company is all in AI - at this point I think just about 100% of the code being merged is AI generated.
In terms of numbers I can see on internal metrics that PRs merged across the company are up by around 33% from the same point last year.
Now, first of all, as a headline figure I would say that's probably less than what CEOs and CTOs and managers are expecting the increase to be.
But let's also not forget some additional factors here:
- Engineers are working more hours due to increased expectations and tighter deadlines
- Incidents are increasing
- Engineers are gaming PR metrics because there's been an increased focus on them (at my company)
- AI generated code probably leads to more code churn
- We're probably spending around $1000-2000 per engineer each month on tokens
And by the way - myself and all of the engineers who joined after the AI coding mandate can't even manually write code in the language we work in if we tried. You might blame us for this but we're under pressure to deliver and we don't have time to manually code to build up the skill of coding in an unfamiliar language. And this isn't just about the language - whatever expertise and knowledge that would have been built up by working on services the old way is no longer being built; I've made PRs to other team's repos and they really have no idea how to review the change - they're in a position where they have to just trust the review bot and give an LGTM review.
Previously after you'd worked in a codebase for a while you'd have an immediate sense of where you'd need to make changes to add some new feature. Now every change is going to require some back and forth with AI to understand where to make the change first - and after you've made the change you probably forgot most of what knowledge came out of the clanker because you didn't really work for it.
When you take all the above into account I'm not sure that going all in on agentic engineering gives any benefit vs much more moderate usage.
And for the record I'm not claiming that AI isn't a great tool - the utility of it is undeniable. What I'm claiming is that 100% AI generated code, particularly in the context of a large/mid sized organisation is not giving clear benefits.
Also I find it quite interesting how AI usage is being driven top down. Traditionally it's been up to individual engineers to decide how they accomplish their tasks. Nobody was mandating what IDE I used. It's pretty clear that CEOs and CTOs are mandating usage because they are scared of being left behind rather than because AI is actually driving meaningful productivity gains. And just a few months ago Uber's COO even said as much with regards to AI productivity:
`Uber COO Andrew Macdonald stated that rising AI spending is getting "harder to justify" because there is no clear link between high token usage and real, useful product gains for consumers.`
I will stop here because you have brain rot from AI usage. Me too.