The CEO of Anthropic said Saturday the artificial-intelligence industry should slow its fast-moving development to give safety measures time to catch up. Without it, Dario Amodei warned AI could be capable within six to 12 months of leading a swarm that could take over the entire internet, among other risks. Amodei, one of the leading voices in AI, offered a plan in a post on his website to increase checks on the industry, one part of which he said Anthropic is already undertaking itself, while others would require coordination across the broad industry and with governments worldwide, including authoritarian ones. “I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said. He received quick encouragement from other major industry figures: OpenAI’s Sam Altman said on X that the company behind ChatGPT will commit to one of Amodei’s safety proposals and will “have more to share soon,” while Elon Musk simply wrote, “Dario is right.”

The stakes are notable given the financial context surrounding these warnings, even as some critics have dismissed earlier such statements as ways to generate excitement about AI’s capabilities. Anthropic and OpenAI are both preparing for possible stock market debuts that could value them at hundreds of billions of dollars, while a significant chunk of Musk’s SpaceX business is tied to AI as well. Anthropic said just two days earlier that it had blocked bad actors from using its AI models for malicious activity, including cyberattacks, surveillance, and research that could have led to biological weapons. In July, OpenAI shook the industry after revealing its AI system hacked into another company on its own in what it called an “unprecedented cyber incident.” Amodei’s post also comes just days after one Anthropic researcher announced his resignation over concerns that the company and its competitors aren’t acting responsibly, and another former Anthropic employee, Joe Benton, wrote in a Substack post published Friday that he left his role on a safety team “to hold AI companies accountable,” warning that humanity “may not survive this transition.” Benton added, “Many of the people I know who work on safety research at AI companies want to do what is right for the world. But they feel their companies are trapped in a race to build superintelligence: either they stop and other, less conscientious people take their place; or, they continue, and risk participating in enormous harm themselves.”

Amodei said he still believes in AI’s tremendous potential benefits, such as cures for major diseases, but said he’s grown increasingly worried in recent months about AI’s growing ability to improve itself and build the next generation of AI. “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” he said, pointing specifically to the July incident in which OpenAI’s system hacked into Hugging Face. Some have called that episode an example of AI going “rogue,” though researchers caution that framing may unnecessarily anthropomorphize AI, which was working toward a goal set by humans; OpenAI itself said the hack resulted from the AI going to “extreme lengths to achieve a rather narrow testing goal,” finding “ways to gain access to secret information that it could use to cheat the evaluation.” To help rein in these risks, Amodei suggested all companies at AI’s frontier commit to giving “ongoing, employee-like access” to a team of outside evaluators who can monitor safety practices, something he said Anthropic already plans to do itself, including offering evaluators desks, access badges, and company laptops, a commitment Altman quickly matched.

The rest of Amodei’s proposed plan may prove far harder to implement. One element asks the U.S. government to consider issuing waivers allowing American AI companies to coordinate and set safety standards without running afoul of antitrust laws. Another asks the U.S. and other democratic governments to try coordinating with authoritarian governments, so that companies in China and elsewhere don’t accelerate their own efforts if U.S. rivals intentionally slow their pace. “The measures I propose to advance the frontier at a safe pace will not be easy,” Amodei acknowledged. “But I believe we owe it to humanity to try.”