Hello fellow skeptics and epistemology nerds.
People are claiming Software Engineers are moving "100x faster". Numbers like these are being used to justify some pretty crazy business decisions. I wanted to know if any of it holds up.
Effectiveness of tooling or practices in software have historically had no reliable research to back it up. Typically justifications of software practices have been built on wobbly towers of deduction. But with LLMs we've started seeing some "reasonable" research being conducted. If you want to skip to the TLDR; then here it is.
Software Engineers using AI (as of about 6 months or so ago) were likely seeing productivity boost of around 20%. Some are more productive than that, but some are slower than they were without AI. 20% is great, but that's 1.2x faster, not 100x.
If you want to full run-down, have a look at the extended article. If anyone knows of any research or details that I have missed, I would love to hear from you.