China intensifies crackdown on illegal AI content

China has taken significant action against illegal and inappropriate AI-related content, removing over 5.61 million pieces of such information, suspending more than 49,000 accounts, and shutting down over 2,400 websites and apps as part of a nationwide campaign, the country’s top internet regulator announced on Wednesday.

Launched by the Cyberspace Administration of China in July, the campaign specifically targeted AI-generated content that spreads false information, violent or vulgar material, impersonates others, or infringes on the rights of children. Local cyberspace regulators were instructed to enhance governance efforts by urging websites and platforms to eliminate illegal and harmful AI-generated content. They were also tasked with taking action against non-compliant accounts and apps to effectively manage the challenges posed by AI-synthesized information, thereby fostering the healthy and orderly development of AI applications.

As part of the crackdown, several accounts on platforms such as Kuaishou, Xiaohongshu, Douyin, and WeChat faced penalties for using AI to fabricate fake disaster scenes and rescue footage, create false social hotspots, publish content promoting feudal superstitions, or use AI-powered digital avatars for deceptive marketing, which misled senior citizens and incited public panic. Other accounts were penalized for employing AI technologies like face-swapping or voice-cloning to impersonate public figures, produce and synthesize vulgar dance videos, and create suggestive images for commercial gain.

Additionally, some accounts were punished for mass-posting AI-generated content featuring horror, grotesque, violent, and vulgar imagery, as well as for creating and distributing videos that mock or demean the images of minors. Following the campaign, the administration stated it will continue to monitor AI development trends and remain vigilant about the risks associated with AI-generated content. It emphasized the importance of cyberspace regulators taking timely and effective measures to prevent AI abuse and sustain efforts to create a better environment for technological innovation.