What grew from the data centre - mechanistic interpretability explained

I've felt for a while that this subject doesn't get as much attention as it should. Perhaps because it is somewhat obscure and complicated.

I feel like it's one of the most important challenges we face, so I did my best to break it down in a way that is hopefully digestible for those who don't follow the challenges of packing vectors into high-dimensional space (fair!).

It's a longish read, but hopefully interesting and thought-provoking. I would be grateful for any feedback.

https://write.as/iain-harper/so-within-so-without-what-grows-from-the-datacentre

u/iainrfharper — 13 days ago

The Winter Spine Race 2015

I ran the Winter Spine race just over 10 years ago and wrote this afterwards. It was quite a different affair then, more bare bones and we had exceptionally bad weather.

I can't recall ever posting it anywhere, although it was adapted in a slightly different format for a book.

Anyway, it's quite a long one, but I figured some of you may enjoy it.

https://iain.so/running-the-spine-race

u/iainrfharper — 16 days ago

Remastering Endtroducing…… for its 30th Anniversary

I’ve seen a few snippets on DJ Shadow’s socials and most recently a short interview with Scott Hansen (Tycho) talking about restoring the original MPC used in the album’s creation.

The album was recorded in kind of a unique way, a product of technical and equipment limitations of the time.

But it sounds like they’re going the whole hog for the 30th remaster so I wrote some speculation based on scant info and my own experience producing with similar setups in the 90s

https://iain.so/break-it-down-baby

u/iainrfharper — 24 days ago

Infinities, impossibilities, and the man in the white linen suit [D]

So I was belatedly reading Matthew Colbrook’s paper on unstable neural networks https://www.pnas.org/doi/10.1073/pnas.2107151119, and the paradox therein sent me back to my university days and reminded me of Kurt Godel.

It kind of amazes me that so few people have heard of him given Einstein clearly thought he was more than an equal.

Anyway, with the default assumption currently being that any problem will yield to more data and compute, it was timely to revisit this. It’s a long read and I’m not sure it’s fully coherent but if you like logic then you might enjoy this and I‘d welcome your thoughts and feedback.

https://iain.so/infinities-impossibilities-and-the-man-in-the-white-linen-suit

reddit.com
u/iainrfharper — 1 month ago
▲ 103 r/SEO

GEO techniques and visibility monitoring is pretty much all snake oil

I’ve worked on SEO my whole career (25 years). So naturally the current paradigm shift is extremely interesting.

So I’ve been going in deep and found that there’s surprisingly little evidence for a lot of what is bandied about. It’s nostalgic in a way; it feels like the early SEO days when nobody really knew anything and there was lots of correlation inference.

For optimisation there are barely any new techniques beyond “do good basic SEO”. The visibility industry is a borderline scam.

Selling visibility scores averaged over invented prompt baskets, queried through API endpoints no consumer uses, weighted by modelled volumes nobody can verify, with woolly methods if they’re published at all is sketchy at best.

But quite a few brand marketers I know are falling over themselves to buy these platforms because they have to report *something* to their bosses.

But the only stable signal is whether you’re in the pool at all, and an afternoon a month with Bing Webmaster Tools and your server logs will tell you that for free.

Maybe age and painful experience have made me cynical!

reddit.com
u/iainrfharper — 1 month ago

Why most agentic search visibility improvement and measurement tools are snake oil

I’ve worked on SEO my whole career (25 years). So naturally the current paradigm shift is extremely interesting.

So I’ve been going in deep and found that there’s surprisingly little evidence for a lot of what is bandied about. It’s nostalgic in a way; it feels like the early SEO days when nobody really knew anything and there was lots of correlation inference.

For optimisation there are barely any new techniques beyond “do good basic SEO”. The visibility industry is a borderline scam.

Selling visibility scores averaged over invented prompt baskets, queried through API endpoints no consumer uses, weighted by modelled volumes nobody can verify, with woolly methods if they’re published at all is sketchy at best.

But quite a few brand marketers I know are falling over themselves to buy these platforms because they have to report *something* to their bosses.

But the only stable signal is whether you’re in the pool at all, and an afternoon a month with Bing Webmaster Tools and your server logs will tell you that for free.

Maybe age and painful experience have made me cynical!

reddit.com
u/iainrfharper — 1 month ago

AI pricing in 2026 is mobile data in 2008 all over again

AI pricing right now feels strangely similar to mobile data circa 2008. Per-token meters, fair-use caps, outcome-based pricing etc etc

It probably ends where mobile ended: flat-rate bundles for most use cases, with small on-device models quietly handming the median workload the way Wi-Fi ate most mobile traffic.

Not necessarily an earth shattering insight, more an amusing comparison for those old enough to remember paying for SMS and ringtones

Full piece here: https://betterthangood.xyz/blog/cost-everything-value-nothing/

u/iainrfharper — 3 months ago