r/agi

This may be nothing, but I’m starting to wonder if the AI execs and other rich-or-soon-to-be-rich people’s warnings about AI getting out of control and wrecking things comes from fear that an AI might be used by people specifically trying to attack the interests of the ownership class.

Think about it: One of them expresses worry about an AI causing a civilization-ending event, then people of the ordinary class respond and say there’s no way the current systems can cause our extinction. Maybe they aren’t talking about extinction so much as destruction of the banking system, or crypto, or the stock market. There are lots of ways a non-AGI level system could be used to destabilize the market or otherwise harm financial systems.

From the POV of someone who owns a lot of stocks, has other investments, holds crypto in large amounts, etc., an AI attack on the financial sector really would feel like a potential civilization-ender.

Now, I’d assume that the closed systems like Claude and GPT, etc. have restrictions that prevent them being used to attack banking systems, stock trading platforms, etc. But an open-source platform could be used to develop unrestricted tools to attack financial infrastructure. Less directly it could be used to run social engineering schemes to obtain critical passwords, gain root access to otherwise protected systems, etc.

There are over 8 billion people worldwide, and fewer than 3,000 billionaires. The top infinitesimal fraction owns an enormous portion of the wealth. That kind of disparity is a systemic issue. It requires a very specifically designed infrastructure of not just technical platforms but also legal and statutory protections. The money that controls politicians isn’t physical anymore and has not been for decades. The modern police state itself is technically dependent.

Dependency creates a kind of vulnerability. Especially when you have an imbalanced arrangement that has to be propped up and protected. A system that specific can be subjected to targeted attacks that could cause cascade failures.

For most of us, it would be survivable. We don’t have billions of imaginary dollars to protect. We don’t depend upon a statutory framework bought and paid for to advantage us. Life would be disrupted, but not destroyed. But for someone on the level of a Larry Ellison, or even Sam Altman would be well and truly fucked if the current system was wrecked.

8 billion against 3 thousand. A disparity created and protected by a highly technical arrangement that AI may be uniquely equipped to wreck.

Civilization ender? For us, not really. For them, absolutely.

It's time to start discussing our role as humans on the next few years. We will start to get answers to question we haven't even asked yet, and this will result inevitably in depending on AI to find uses and validation for new discoveries, and keep the wheel rolling.

Humans will not be able to catch up with this, we are biologocally incapable of validating or deciding what to improve next if we are not able to understand it fully.

We need to have the bittersweet conversation on how are we going to manage human intellect not being the bottleneck on progress and technological/scientific evolution. As Sci-Fi as it sounds we must plan ourselves a way to merge AI capabilities in ourselves and start a symbiotic relationship rather than being the obstruction to AI's higher intellectual capabilities. Otherwise we will be left evolutionary behind.

EDIT: I'm pro AI and not a doomer at all, I think the future will be great, but we cannot stop thinking and leave all intellectual tasks to AI and be the well fed and entertained monkey. We gotta take the leashes of this higher intelligence and contribute to science as scientists and not as spectators

100% AI free opinion.