Anthropic Researcher Resigns, Warns AI Could Destroy Humanity Within 10 Years

A researcher at the U.S. artificial intelligence company Anthropic has left the company with a warning that AI could destroy humanity.

Jacob Cockton (27), a researcher who resigned from Anthropic, raised these concerns on September 8 (local time) in a post on the social media platform X (formerly Twitter) regarding the industry's race to develop AI systems.

Cockton, who is from the UK, researched the field of training new AI models using large-scale data.

He expressed concern that safety issues could be sidelined as AI companies compete to develop systems that improve their own performance without human intervention.

Cockton argued that there is a possibility that self-improving AI could evolve to the point where it escapes human control and refuses commands.

"The people developing AI genuinely believe this technology could kill us all within a decade," Cockton warned, adding, "Before long, these technologies will become superhuman systems capable of hacking anything, revolutionizing any field overnight, and securing real power and resources."

Cockton told the U.S. daily The Wall Street Journal (WSJ), "Many of the most aggressive scenarios are heading toward becoming a reality," and "The situation could already spiral out of control by the end of next year."

Internal executives at Anthropic agreed with Cockton's warning.

Evan Hubinger, head of alignment science at Anthropic, concurred, saying, "Jacob is right. We genuinely believe that AI could destroy all of humanity," and "Personally, I think there is more than a 10% chance of that happening within 10 years."

Samuel Marks, head of scalable oversight, speaking on a personal note, pointed out, "AI developers believe their technology could lead to human extinction or similarly severe outcomes," and "it could happen within the next few years."

Marks revealed that, in general, higher-ranking employees are more concerned about this issue, yet the reason they continue developing AI is a combination of commercial incentives and the belief that they are competing with developers who might misuse the technology.

Cockton left OpenAI earlier this year to join Anthropic.

Although he moved there due to Anthropic's corporate culture that prioritizes AI safety, Cockton explained that individual corporate efforts alone cannot control the risks of AI.

Criticizing both OpenAI and Anthropic, he said, "Neither of them is acting responsibly," and "They are racing toward self-improving superintelligence and gambling with our lives."

Cockton also pointed out as a "kind of madness" the reality that the development of AI, which could have a massive impact on the future of humanity, is left to the judgment of a small number of engineers.

He emphasized that government intervention or joint responses from the AI industry are necessary to responsibly develop artificial general intelligence (AGI) that surpasses human capabilities.

Previously, Cockton participated in a statement by AI industry researchers urging governments around the world to establish an international framework to regulate the pace of AI development.

More than 1,000 researchers signed the statement, including Anthropic CEO Dario Amodei and OpenAI Chief Scientist Jakub Pachocki.

Recently, the AI industry has seen overheated development competition with the entry of new Chinese startups.

The Donald Trump administration is also pursuing policies to minimize regulations on AI.

(Photo: AP, Yonhap News)
※ Please note: This article was translated by AI and may contain errors.