OpenAI bans some Chinese users from using ChatGPT for social media monitoring

OpenAI

OpenAI has announced that it has banned a set of accounts originating from China from using its ChatGPT service. The accounts were found to be involved in activities related to social media monitoring, according to a threat intelligence report released by the company.

The report highlights OpenAI's efforts to identify and prevent the misuse of its AI models, including actions taken against authoritarian regimes. The banned accounts were generating descriptions for a social media listening tool. This tool was claimed to provide real-time reports about protests in the West to Chinese security services.

"The operators used our models to proofread claims that their insights had been sent to Chinese embassies abroad, and to intelligence agents monitoring protests in countries including the United States, Germany and the United Kingdom,” said OpenAI.

OpenAI's investigation revealed that the users were also using the models to debug code for this tool. Although the tool itself appeared to be powered by a non-OpenAI model, the activity was a clear violation of the company's policies against misuse. “Our policies prohibit the use of AI for communications surveillance, or unauthorized monitoring of individuals. This includes activities by or on behalf of governments and authoritarian regimes that seek to suppress personal freedoms and rights,” said OpenAI in the report.

In a separate incident, OpenAI also banned a ChatGPT account that was generating comments critical of Chinese dissident Cai Xia. These comments were then posted on social media platforms. The same actor was also found to be using the service to generate anti-US news articles in Spanish. These articles were subsequently published in Latin American news outlets.

“This is the first time we’ve observed a Chinese actor successfully planting long-form articles in mainstream media to target Latin America audiences with anti-US narratives,” said OpenAI