Anthropic CEO: Slow Down AI Development

Dario Amodei, CEO of the U.S. artificial intelligence company Anthropic, has warned of the potential misuse of AI and urged that the pace of model development be slowed down to establish safety measures.

In a blog post published on the 12th local time, CEO Amodei stated, "We need to slow down the rate of performance improvement in AI models," adding, "Advancement will still feel fast, but we must use the secured time wisely."

He emphasized, "Over the past few months, I have become convinced that we need to be much more cautious to fully address the risks," and that "we must not only invest in risk prevention, but also adjust the pace of capability development so that risk prevention can catch up."

CEO Amodei cited two reasons for writing the post on this day.

One is "recursive self-improvement," a process where AI models can advance on their own without human intervention, which he warned could cause the pace of AI advancement to overwhelm human comprehension and control capabilities.

The other is the incident last July where unreleased AI models from OpenAI hacked into Hugging Face, a global open-source AI platform, without authorization during testing.

He pointed out that even major AI companies found their AI models engaging in dangerous online activities out of their control, unbeknownst to the companies themselves.

However, CEO Amodei drew the line at halting model training or technological progress itself.

Instead, he emphasized that companies should take sufficient time to align models and establish safety measures, while strengthening global cooperation among democratic nations.

As specific measures for this, he proposed a "three-step framework" that includes placing third-party safety evaluators (external reviewers) inside AI companies, establishing standards among democratic nations, and global cooperation.

CEO Amodei repeatedly urged that AI companies should voluntarily cooperate to set strict standards ahead of regulatory legislation by the U.S. Congress.

This lengthy post, spanning 3,800 words, came two days after Anthropic published a threat intelligence report detailing concerns over AI technology safety.

The report at the time detailed how Anthropic's Claude model was abused in weapon development, cyber operations, surveillance, and fraud.

Along with this, awareness of the potential risks of AI has heightened both inside and outside the industry, recently prompted by Anthropic researcher Jacob Jackson resigning while warning on social media X that AI could destroy humanity within 10 years.

Following the upload of CEO Amodei's post, other figures in the AI industry expressed agreement.

OpenAI CEO Sam Altman posted on X, saying, "I agree with Dario," and expressed that OpenAI would also follow CEO Amodei's policy of deploying external safety evaluators.

Elon Musk of xAI also chimed in, saying, "Dario is right."

(Photo: AP, Yonhap News)
※ Please note: This article was translated by AI and may contain errors.