Italy’s ChatGPT Block: Privacy protection concerns stalk OpenAI and other Generative AI developers

Generative AI chatbots like ChatGPT have disrupted the technology industry as we know it. It will now be pertinent to ensure that it also protects the principles of data privacy and security in its functioning.

ChatGPT has been in the news, this time around, for its compliance, or the lack of it, with data protection laws worldwide, specifically the European Union-General Data Protection Regulation (EU-GDPR).  Last week, Italy became the first Western country to temporarily block the advanced chatbot “with immediate effect” over concerns relating to privacy like opacity in ChatGPT’s use of personal data and its non-compliance with EU- GDPR provisions.

As businesses, organisations, and corporations around the world transition towards automated processes for analysing market trends, generating reports, customer service etc, it is becoming increasingly important to ensure that new technologies are compliant with established data protection laws.

Distinguish Personal, Non-Personal Data

Generative AI is a subset of artificial intelligence that works by training algorithms with large datasets that are used by the system to generate new data. ChatGPT is an AI chatbot that was initially built on a family of large language models (LLMs). The platform is known to understand and generate human-like answers to text prompts since it has been trained on huge amounts of data.

It is suggested that the dataset which was used for training ChatGPT has billions of words and images scraped from various places like websites, articles, blog posts, etc. The dataset used is publicly available information, which includes both personal and non-personal data.

While personal data warrants protection and security, non-personal data unlocks value benefiting individuals, businesses, and communities. Therefore, blocking/banning generative AI solutions like ChatGPT just on the premise of personal data breaches and mishandling could be disproportionate. Having said that, it is also necessary to ensure that Generative AI solutions enable the secure utilisation of personal data keeping purpose limitations, contextual integrity, and appropriateness intact.

The reinforcement learning algorithm instilled in the model constantly learns and updates itself for appropriate responses based on the nature of prompts. The lack of feedback mechanism within the human-centred AI systems has been one of the key concerns, and usage of RLHF (Reinforcement Learning from Human Feedback) promises the possibility that ChatGPT could evolve into a Responsible AI tool through incorporating feedback from human interaction.

Italy’s Concerns

The key concerns cited by the Italian watchdog include non-compliance with the provisions of EU-GDPR by OpenAI failing to give information to users/people whose personal data has been utilised to train the language model about the processing of their data.

Consent is the bedrock on which not only the EU-GDPR, even India’s Digital Personal Data Protection Bill sits, where it mandates that personal data shall be processed only after obtaining consent from data principles at the commencement of its processing. However, the consent-based approach doesn’t consider the complex data processing mechanism in the case of generative AI like ChatGPT.

Besides, this could also cause a fall through the cracks as the determining legitimacy of consent is nebulous in Generative AI operations. However, there has to be means through which individuals’ agency over their personal data used for training the algorithms across the data lifecycle is protected and ensured.

Bans No Solution

Moreover, the Right to be Forgotten/Right to Erasure as enshrined in Article 17 of the GDPR is difficult to enforce with respect to ChatGPT given the persistent nature of the data created by these systems. The application uses natural language processing to create responses from the collected data, making it nearly impossible to remove all traces of an individual’s personal information.

This necessitates a thorough understanding of how AI systems interpret and generate responses to enable the complexity of deleting data, if required. Generative AI chatbots like ChatGPT have disrupted the technology industry as we know it. It will now be pertinent to ensure that it also protects the principles of data privacy and security in its functioning.

Blanket banning or blocking might not be the answer everywhere given the huge potential of AI models to transform different sectors, it will be important to institutionalise the right checks and balances to ensure that the disruption that this technology causes is constructive. ​

A coordinated regulatory approach is required incorporating the responsible use of AI including accountability, human intervention, security, bias prevention, accuracy and syncing of automated decision making with privacy principles.

Laws regulating AI, which are in the making in countries like the EU, US and Brazil, must keep the above considerations in mind. The laws must aim to regulate the impact of AI on society and fight the biases of generative AI algorithms and software.

Jameela Sahiba is Senior Programme Manager, The Dialogue. Kamesh Shekar is Programme Manager, The Dialogue, and leads the data governance vertical. Views are personal and do not represent the stand of this publication.