AI-detectors biased against non-native English speakers, says Wharton Professor

Those who can write effective prompts can evade detection more effectively, according to the study's findings. (Photo by Tara Winstead/Pexels)

Artificial intelligence (AI) detectors for spotting content generated by technologies such as ChatGPT should not be used in classrooms, according to a professor at the Wharton School of University of Pennsylvania. These detectors, which were reported to being used by teachers across the world including India, seem to be hugely biased against non-native English speakers.

“You really, really should not be relying on AI detectors for classroom use,” tweeted Ethan Mollick, Associate Professor of innovation and entrepreneurship at Wharton School.

Also read: ChatGPT creator Sam Altman meets another CEO in India. This one is his friend from Stanford

Citing research done by Stanford University, Mollick tweeted, ”This new paper shows that not only are they very easy to defeat by just prompting a couple of times, but they have insane false-positive rates against non-native English speakers.

The study titled “GPT detectors are biased against non-native English writers” evaluated the performance of widely used GPT detectors using writing samples from native and non-native English writers.

“Our findings reveal that these detectors consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified,” stated the study.

“Furthermore, we demonstrate that simple prompting strategies can not only mitigate this bias but also effectively bypass GPT detectors, suggesting that GPT detectors may unintentionally penalize writers with constrained linguistic expressions,” stated the study.

The researchers called for a “broader conversation about the ethical implications of deploying ChatGPT content detectors” and cautioned against the detectors’ use in evaluative or educational settings, “particularly when they may inadvertently penalize or exclude non-native English speakers from the global discourse”.

Also read: TikTok-owner tests ChatGPT-style bot after joining China AI race

Mollick gave other reasons why these detectors don’t work: “1) They are often trained on GPT-3.5, so GPT-4 beats them 2) Even if they alert you to potential AI use there is no way to see if that is true 3) Students working interactively with AI defeat tests, as my class found”.

He added the dangers of the detector AI making up stuff. “Also, never ask an AI to identify whether something is written by AI. It cannot do it, but might make up an answer randomly if pushed enough,” posted Mollick.