China Steps Up Crackdown on AI Impersonation, Removes More Than 47,000 Illegal Posts

Chinese authorities have removed more than 47,000 pieces of unlawful or rule-breaking online content and taken action against nearly 4,000 accounts accused of using artificial intelligence to impersonate public figures, as Beijing intensifies efforts to regulate the misuse of generative AI on social media platforms.

The Cyberspace Administration of China (CAC) said on Nov. 6 that it had instructed online platforms to conduct a targeted campaign against accounts using AI-generated images, voices and videos to imitate well-known individuals. The accounts were allegedly used to promote products, attract followers, spread misleading information and generate commercial traffic.

According to the regulator, more than 47,000 items of illegal or non-compliant information were removed, while 3,961 accounts were dealt with during the campaign.

The CAC identified several categories of misconduct. These included unauthorized use of public figures' likenesses and voices in commercial promotions, fabrication of rumors using manipulated images, impersonation intended to attract followers, and the creation of vulgar or misleading content featuring well-known personalities.

Some accounts reportedly used AI-generated voices and altered images to make it appear that celebrities were endorsing products. Others published fabricated statements or personal stories in the first person, creating the impression that the public figures themselves were communicating with viewers.

The regulator also highlighted cases in which public figures' images were used to spread false claims about their health or personal lives. Such content can mislead audiences and potentially damage reputations, particularly when it is distributed widely through short-video platforms and social networks.

The campaign reflects a growing challenge for online platforms as generative AI tools make it easier to create convincing synthetic media. Technologies that can reproduce a person's voice, facial appearance or speaking style have become increasingly accessible, allowing users with limited technical expertise to produce realistic-looking videos and audio recordings.

While AI-generated content can be used for legitimate entertainment, education and commercial purposes, unauthorized impersonation raises questions about consent, intellectual property, consumer protection and the spread of misinformation.

The use of synthetic endorsements is particularly sensitive in e-commerce. Consumers may make purchasing decisions based on the perceived recommendation of a celebrity, influencer or professional expert. If that endorsement is fabricated, the commercial transaction may be based on misleading information.

The CAC said the latest campaign was intended to protect the legitimate rights and interests of internet users and create a more orderly online environment. It also called on platforms to strengthen their responsibilities for content management and to improve measures against the misuse of AI technologies.

China has been developing a regulatory framework for generative AI and synthetic media. Its existing rules include requirements for providers of generative AI services and regulations requiring AI-generated content to carry identifying labels. The framework is designed to increase transparency and make it easier for users to distinguish synthetic material from authentic content.

However, enforcement remains difficult because AI-generated material can be reposted across multiple platforms, modified to avoid detection or distributed through accounts that are quickly replaced after being suspended.

The regulator's latest figures demonstrate the scale of the problem but do not provide a complete breakdown of the platforms involved, the number of individual victims or the financial losses associated with the identified cases.

The crackdown may also increase compliance responsibilities for social media companies and online marketplaces. Platforms could face greater pressure to verify commercial endorsements, respond to impersonation complaints and detect manipulated audio and video before they reach large audiences.

The announcement was reported by Xinhua, while the Cyberspace Administration of China published examples of the accounts and content targeted by the campaign.

As generative AI becomes more deeply integrated into online commerce and social media, the effectiveness of China's enforcement efforts will depend on whether platforms can identify impersonation quickly while protecting legitimate creative and commercial uses of the technology.