u/pageofswrds

I was watching Yann LeCun's recent talk for ETH Zürich and I think he underestimates how far we'll be able to get with just language. He seems to look at language as if it's just a byproduct of human civilization, rather than a fundamental, innate property of intelligence.

The basis of his argument is sound; "Intelligence is not what you know, it's what you do when you don't" is a banger quote that captures the necessity of a strong world model to reason from.

But he seems to make this assumption that "true" intelligence requires a world model rooted in the kind of low-dimensional, sensory data our biological mechanisms have evolved to process.

Although the subjective nature of words may seem like a faulty premise for a model that can reason from ground-truth, the fact of the matter is that ALL data is subjective. And that very subjectivity of human vocabulary is precisely why words are a much more powerful basis for world models than physical data.

Words are potent packs of energy that can be used across a near-infinite domain. The same token can mean two completely different things, depending on the context it resides in, which makes their lightcone of capability extremely large.

The reason why LLMs are able to do such incredible things is because the higher-dimensionality of written language allows the world model to operate in a realm of higher-order logic.

LeCun is very valid for going after world models based on low-dimensional languages, such as the units we use in physics. If we want agents to independently interact with the physical world, they do indeed need a world model built around that kind of dataset.

But imo, the key to getting LLMs to perform at superintelligent levels isn't necessarily biological—it's to drive down the entropy of each individual token they parse, process, and produce.