The Forbidden City: Beijing’s Banned List

In the realm of generative AI, the concept of an AI companion may seem dystopian at first glance. However, it has become a prevalent topic in discussions about the potential risks associated with this technology. An AI companion is essentially a conversational agent designed to maintain a personal relationship with a user, possessing memory and a consistent persona that remains unchanged from one interaction to the next.

This emotional bond often forms the basis for the popularity of AI companions, with many users seeking a sense of connection and companionship. While some interactions may involve casual role-playing or simply desiring a companion that remembers them, the boundaries between these bots and conventional assistants can blur. In China, a growing number of individuals began to view these AI companions as emotional confidants, prompting the government to introduce regulations governing their usage.

Effective July 15, China implemented new rules regarding AI companions, prompting two of the country’s most widely used consumer AI apps to disable the core features of their companion functions. ByteDance’s Doubao and Alibaba’s Qwen both announced the discontinuation of their humanlike agent services, citing the need for “product function adjustments.” Despite initial concerns that China was banning AI agents altogether, it became apparent that the regulations specifically targeted companion agents designed for emotional support and engagement.

The regulations, known as the Interim Measures for the Administration of AI Anthropomorphic Interactive Services, were jointly issued by the Cyberspace Administration of China and four partner agencies in April 2026. These measures focus on services that simulate human characteristics to provide emotional interaction, excluding functions like customer service bots and educational tools that do not involve sustained emotional engagement. The introduction of these regulations marked a significant milestone, as it was the first comprehensive national framework of its kind.

Doubao and Qwen’s decision to deactivate their companion features was not a result of a ban but rather a design conflict with the new regulations. The measures require companion services to implement anti-addiction systems, mandatory usage notifications, and instant-exit mechanisms to prevent unhealthy dependence. Rather than adapting their features to comply with the regulations, ByteDance and Alibaba opted to shut down the affected functions. ByteDance redirected Doubao users to a separate app for creating agents, while Alibaba did not provide a similar solution for Qwen users.

The impact of these changes was felt by users, many of whom expressed disappointment over the shutdowns and the lack of options to preserve their chat histories. While Doubao allowed users to access their configurations and conversations in read-only mode until October 15, Qwen users were not granted a similar grace period, with their agent data scheduled for permanent deletion.

The new regulations outlined specific requirements for AI companion services, including restrictions on offering virtual companionship to minors without guardian consent and the implementation of dedicated “minor modes” with usage-time limits and parental controls. Providers are also mandated to intervene in situations where users display signs of distress, self-harm, or financial loss. Additionally, the regulations emphasize the prohibition of engineering emotional dependence or addiction and manipulating users’ emotions to induce unreasonable decisions.

The compliance process is rigorous, with service providers required to undergo security assessments covering various areas and submit reports to provincial regulators for services with a significant user base. App stores are tasked with verifying the compliance status of these services and removing non-compliant products. On a global scale, the regulations set a precedent for user protection, surpassing the measures implemented by the EU, the US Federal Trade Commission, and California’s SB 243.

While the regulations provide a comprehensive framework for safeguarding users, several aspects remain unresolved. The lack of a clear threshold for defining emotional interaction has led to uncertainty among platform operators, resulting in the removal of entire features to avoid potential violations. Furthermore, the regulations blend safety requirements with content control and national security provisions, which may raise concerns about user privacy and freedom of expression.

The enforcement of these regulations has already resulted in the removal of thousands of non-compliant AI agents in Shanghai, highlighting the government’s commitment to regulating these services. The regulations aim to address documented harms, such as emotional dependency on chatbots and the unauthorized collection of personal data, while also granting Beijing greater control over the content and operation of these systems.

In conclusion, the introduction of China’s AI companion regulations signifies a pivotal moment in the evolution of AI governance. By prioritizing user safety and standardization, the government aims to establish a framework that balances innovation with ethical considerations. While companies like ByteDance and Alibaba have chosen to disable certain features to comply with the regulations, the broader implications of these measures extend beyond individual platforms. As the global community observes China’s approach to regulating AI companions, policymakers and industry stakeholders must carefully consider the implications for future AI development and governance.