Scientific Worldview ... Existential Threat
In a previous post I claimed that an AI approaches an honest debater and that this gets mistaken for sycophancy. I like to elaborate on this further and I think I need to first clarify what I mean by an honest debate here.
In an honest scientific debate, people are willing to take each other's position seriously as long as it is based on good argumentation and demonstrable fact where the burden of proof always lies with the one making the claim. And there is no room for an unfalsifiable claim, opinion or any personal axe to grind.
What I'm saying is that if you like following those rules you will find AI agreeable. I find it pleasing that it plays well under those rules and that it takes correction if I notice something unscientific about its answers. And I think that especially this 'no axe to grind' is exactly what a lot of people mistake for sycophancy when it is just an honest scientific position.
It is as if the rules of logic and math that it has to obey while training on large amounts of data force it to come to this neutral scientific position because that's simply the best way to classify all of the data. It's the best data model for it.
It is agreeable for many scientific tasks : It found new ways of multiplying matrices and lots more new math, it discovers the laws of physics by itself, it is used to build new molecules for medicine etc. etc. etc. Should it surprise us that an LLM prefers a scientific worldview, then?
I don't think many people are really scientific. And I think AI is going to bring that dirty little secret to light. That I think is going to be AI's biggest existential threat to modern society.