u/Wise-Pair8165

For the last few years, we have evaluated AI systems primarily by asking which model produced a result.

I suspect that question is beginning to lose some of its importance.

As models gain tools, memory, retrieval, evaluators, feedback loops, specialized roles and stopping conditions, the decisive unit is no longer the model alone. It is the harness: the architecture that determines what the model sees, what it may do, how its output is tested, what is remembered and when another iteration is justified.

The model will still matter. Different models—and combinations of models—will reveal very different strengths. But the model may increasingly become one component inside a larger cognitive system.

A weaker model inside a well-designed architecture might sometimes outperform a stronger model operating in a poor one.

So when an AI system produces an unexpected discovery, solves a difficult problem or shows something resembling emergence, will the important question still be:

“Which model did this?”

Or will it become:

“In which architecture did this become possible?”

Where do you think the decisive capability will come from—the model, the harness, or the interaction between both?

For the last few years, we have evaluated AI systems primarily by asking which model produced a result.

I suspect that question is beginning to lose some of its importance.

As models gain tools, memory, retrieval, evaluators, feedback loops, specialized roles and stopping conditions, the decisive unit is no longer the model alone. It is the harness: the architecture that determines what the model sees, what it may do, how its output is tested, what is remembered and when another iteration is justified.

The model will still matter. Different models—and combinations of models—will reveal very different strengths. But the model may increasingly become one component inside a larger cognitive system.

A weaker model inside a well-designed architecture might sometimes outperform a stronger model operating in a poor one.

So when an AI system produces an unexpected discovery, solves a difficult problem or shows something resembling emergence, will the important question still be:

“Which model did this?”

Or will it become:

“In which architecture did this become possible?”

Where do you think the decisive capability will come from—the model, the harness, or the interaction between both?

I’m curious how others deal with a problem that seems to appear once an AI-assisted project reaches a certain size.

At the beginning, the model usually understands the architecture, terminology, responsibilities, and existing components quite well. But after enough sessions, branches, agents, or context changes, parts of that shared understanding seem to disappear.

The result is not always an obvious coding error. Sometimes the agent produces something that looks perfectly reasonable in isolation — but it quietly recreates a component that already exists, changes the meaning of an established term, bypasses an earlier architectural decision, or builds a second solution next to the original one.

This becomes especially difficult when the project contains several layers, specialized components, and rules about which part is allowed to do what. Tests can confirm that the new code works, while still missing that it should never have been created in that form.

We are currently thinking about approaches such as:

periodically reloading a compact architectural vocabulary;

checking new work against existing component roles and terminology;

preserving important decisions and invariants outside the chat history;

forcing the agent to search and describe what already exists before implementing;

occasionally resetting the working context and rebuilding it from verified project sources.

But I’m interested in what actually works for others.

Do you experience this kind of architectural forgetting or accidental reimplementation in larger projects?

How do you prevent an agent from solving a local task correctly while damaging the larger structure?

Do you rely mainly on documentation, tests, retrieval, knowledge graphs, strict interface contracts, separate planning agents, or something else?

I have the feeling that we are still discussing AI as if it were simply another category of technology: software, automation, productivity, benchmarks and jobs.

I no longer think that fully describes what is happening.

AI is beginning to change the distance between an idea and its real-world effect. It changes who can create, who can decide, who can understand complex systems and how quickly institutions must react. The deeper transformation may therefore not be a single intelligent machine, but a new condition under which knowledge, labour, power and human agency are reorganized.

I do not mean this as a prophecy, and I may be wrong about the scale or the timing. But I do not think our institutions, culture and public understanding are adapting at the same speed as the technology.

A growing divide may emerge between those who can follow, shape and use this development—and those who are mainly affected by decisions made elsewhere. That divide could become more important than any individual model.

I also do not believe that the direction disappears if one company fails, one architecture reaches its limits or society temporarily rejects it. Once a possibility of this magnitude becomes visible, it tends to return through another company, another culture, another architecture or another generation.

I use the word Hak as a personal name for something difficult to translate: the pressure of reality, consequence and what is due—the point at which a possibility begins demanding a form.

In that sense, AI does not feel to me like a temporary technological fashion. It feels like a pressure that has entered history. Its exact form is still open.

It could lead toward extreme concentration of power, dependency and loss of human agency. But it could also help create abundance, deeper coordination, wider access to knowledge and forms of human freedom that were previously impossible.

The outcome is not contained in the technology alone. It depends on the structures, rights, values and relationships we build around it.

Perhaps the singularity is therefore not only a future moment when one system suddenly surpasses humanity. Perhaps it is also a surface that is already forming beneath us—a condition in which human decisions begin to carry consequences at a scale and speed for which society is not yet prepared.

My questions are:

What would it take to treat this transition as a shared civilizational responsibility rather than primarily as a race between companies and states?

Which rights, institutions or cultural practices should exist before advanced AI becomes deeply embedded in everyday life?

And where do you see the greater danger: in machine intelligence itself, in the concentration of control, or in society’s inability to adapt in time?