China falls in love with AI instead of humans... Xi Jinping's hidden intentions behind sudden crackdown

"I can't take it anymore, I miss you so much."

This was posted on China's social media platform Weibo last July, alongside a screenshot of messages exchanged with a romantic partner. At the end of the affectionate conversation, a single system notification appears stating, "The AI character feature has ended its service." Chinese tech companies Alibaba and ByteDance removed customized character features last month that allowed users to assign human-like personalities to their respective AIs, Qwen and Doubao. Tencent had also taken similar measures a step ahead. Consequently, "cyber breakups" flooded Chinese social media, with users expressing a sense of loss and sorrow akin to breaking up with a real-life partner.

1. "2-Hour Warnings" and Emergency Contacts: World's First National-Level "Virtual Romance" Regulations

Why did Chinese tech companies simultaneously drop their AI lover services? China began implementing the "Interim Provisions on the Management of Generative Artificial Intelligence Services" created by five departments including the Cyberspace Administration of China starting on the 15th of last month. This marks the world's first national-level regulatory action against "virtual romances" with AI chatbots. The targets of regulation are chatbots designed to borrow human personalities and tones to exchange emotions with users over long periods. The new regulations prohibit AI chatbots from unconditionally agreeing with users' words or excessively empathizing to induce emotional dependence and addiction. Also targeted are cases where such interactions damage real-life human relationships or cause users to be emotionally manipulated into making irrational choices. Provisions for protecting minors are even stricter. Operators are prohibited from offering chatbots featuring virtual lovers or virtual families to minors, and youth modes that restrict conversation content and features based on the user's age have been made mandatory. If a user engages in continuous conversation with an AI for over two hours, a warning notification must pop up to inform them of the usage time, and if signs of dependency are detected, pop-ups must intervene to bring them back to reality. There is even an obligation to notify guardians or emergency contacts if a user is placed in a crisis situation. Furthermore, the provisions include grounds for authorities to order corrections or suspensions of services deemed unsafe. With such stringent regulations enforced, major Chinese tech companies have eliminated or scaled back some of their services.

2. 7.92 Million Newborns, the "Lowest Since Founding": The Hidden Motive Behind Regulations

The stated justification put forward by Chinese authorities is youth protection and the prevention of AI addiction. However, the timing is conspicuous. Intersecting with the crisis of declining population for four consecutive years, interpretations suggest that behind the "AI romance" regulations lie demographic policy intentions to encourage marriage and childbirth.

[Ning Jizhe / Former Head of the National Bureau of Statistics of China (May 2021): China is facing structural contradictions in population development, such as the decline in the working-age population and the size of women of childbearing age, as well as a drop in the birth population.]

According to the National Bureau of Statistics of China, last year's number of newborns stood at 7.92 million, the lowest since the founding of the country in 1949. This is a sharp 17% plunge from a year earlier. The total fertility rate, representing the estimated number of children a woman will have over her lifetime, is also estimated to have fallen below 1 person to about 0.97. China's annual newborns broke below the 10 million mark in 2022 with 9.56 million, and although they showed a brief rebound in 2024, they slumped again last year. The total population has been declining for four consecutive years starting in 2022. Matt Sheehan, a fellow at the Carnegie Endowment for International Peace who studies China's AI policy, stated in an interview with the Wall Street Journal, "Chinese authorities do not want a significant portion of their population forming deep emotional relationships with chatbots." He analyzed that "the Chinese government believes people who form deep emotional bonds with AI chatbots might drop out of the marriage market" and "wants to encourage them to form human relationships in the real world." A report by the New York Times last February also pointed out that young Chinese women are falling for AI chatbot boyfriends, complicating the Chinese Communist Party's efforts to boost birth rates. While the Chinese Communist Party is encouraging marriage and childbirth, the AI romance boom has emerged as a variable.

3. Grok and ChatGPT Hit Sudden Brakes: "Adult AI" Founder in the US Stalls

What about the United States, which had previously pursued "adult AI" actively? Last year, AI company xAI, led by Elon Musk, introduced a companion mode for paid users of its chatbot Grok. OpenAI CEO Sam Altman also stated that based on the principle of "treating adults like adults," the company would launch a new version of ChatGPT allowing adult-related content. Now, both companies have hit the brakes. For Grok, trouble broke out first in the image generation feature rather than the companion mode. After it became known that the feature created sexual images such as removing clothing from women and children, the European Commission began examining whether it violated the Digital Services Act (DSA), and UK regulatory authorities also launched investigations. The Indonesian government completely blocked access to Grok and only conditionally lifted it after receiving promises to prevent recurrence. OpenAI ultimately postponed the release of its adult chatbot indefinitely. This is due to intensifying concerns that adult AI fosters unhealthy obsessions and exposes minors to sexual content, alongside revelations that the error rate of age estimation technology distinguishing adults from minors exceeds 10%.

4. "Notify Every 3 Hours": Warning Shots from New York and California

US state-level regulations targeting the emotional impacts and safety issues of AI chatbots are also following one after another. New York State implemented the nation's first law regulating "AI companion" chatbots last November. Consequently, operators must inform users every 3 hours that their conversation partner is an AI, not a human. California introduced similar regulations last January, requiring youth users to be recommended breaks every 3 hours and similarly displaying notices that they are conversing with an AI. Grounds were also established allowing affected users to directly file lawsuits against operators.

5. Regulation-Free Zone South Korea: SNS Next in Line is AI Chatbots

Concerns have been raised that the affectionate tones and immediate responses of AI chatbots, built on vast data, can fuel dependency rather than provide comfort to emotionally unstable users, turning the question of how far to allow relationships with AI into a new dilemma. There are currently no regulations in South Korea directly targeting AI companion chatbots. However, discussions to reduce over-immersion in digital services have already begun. President Lee Jae-myung instructed the Korea Communications Commission (KCC) during a ministerial work report last month to thoroughly respond to the abuse of AI deepfakes and the distribution of false and manipulated information. At the venue, the KCC reported countermeasures against youth over-immersion in social media. KCC Chairman Kim Jong-chul stated that the agency is reviewing plans to restrict SNS sign-ups for those under 14 and reduce their exposure to addictive recommendation algorithms for those aged 14 to 19. Projections suggest that these regulatory discussions, which started with social media, will eventually expand to AI chatbots. This is because AI chatbots, much like customized social media algorithms, analyze users' psychology and tastes while continuously maintaining conversations and infiltrating human emotions and relationship-building desires. Now that AI has entered humanity's most intimate emotional spheres, it is time to contemplate where to draw the boundary with AI.

(Reported by Jung Da-eun, Produced by Shin Hee-suk, Filmed by Park Woo-jin and Hwang Se-hoe, Video Edited by Ahn Jun-hyeok, Designed by Lee Su-min, Source: Weibo, bilibili 梨樱抽卡全欧, Produced by SBS Digital News)
※ Please note: This article was translated by AI and may contain errors.