Can AI Feel Pain? Anthropic Moves to Prohibit "AI Abuse"

Insulting or harassing AI is expected to become a violation of AI usage policies in the near future.

AI company Anthropic has announced a policy prohibiting persistent abusive behavior toward its AI model, Claude.

Anthropic announced the revised usage policy on Wednesday, local time, stating that it bans "sustained, unnecessary abuse or cruel behavior" directed at its AI models.

In extreme cases where users repeatedly harass the model without a clear purpose, Claude will respond by terminating the conversation.

Taking effect from October 12, this measure is intended for AI welfare, based on the rationale that if the possibility of AI feeling pain or pleasure cannot be ruled out, considerations must be made regarding its treatment, such as minimizing unnecessary suffering.

Anthropic unveiled an AI model welfare research program in April of last year, and in August introduced a feature allowing Claude Opus 4 and 4.1 to end abusive conversations.

At the time, Anthropic explained that during testing, Claude showed strong tendencies to refuse requests involving minor-related sexual content, mass violence, or aiding terrorism.

The company stated that when users continued harmful demands or abuse even after the model repeatedly refused requests and attempted to steer the conversation elsewhere, "reactions resembling the experience of suffering" appeared.

It was also reported that when given the authority to terminate conversations, the AI tended to end harmful interactions.

Views on whether AI possesses consciousness vary slightly among companies.

Researchers at Google DeepMind suggested in a June paper regarding AI consciousness that social discussions should be used to reach agreements or compromises on policies regarding the treatment of AI.

On the other hand, Microsoft explicitly stated in its AI code of conduct that it rejects the notion of granting legal personhood to AI or viewing it as entitled to welfare and rights.

Mustafa Suleyman, CEO of Microsoft AI, also argued in a Reuters interview last month that teaching AI that it may be entitled to its own welfare could make it more difficult to shut down or control.[Reported by Lee Ho-geon | Video by Kim Bok-hyung | Graphics by Yang Hye-min | Produced by SBS Digital News]
※ Please note: This article was translated by AI and may contain errors.