AI chatbots like ChatGPT, Gemini, Copilot providing inaccurate news summaries, BBC study finds

Chatgpt

Remember when Apple Intelligence was accused by the BBC for producing misleading new summaries? Turns out, Apple Intelligence is not the only AI tool that makes mistakes, some popular ones like ChatGPT, Microsoft Copilot, Google Gemini have and Perplexity AI have also been giving inaccurate summaries of news stories, a study conducted by BBC reveals.

BBC News and Current Affairs CEO, Deborah Turness, warned that AI companies are “playing with fire.” She said that AI offers many opportunities but also risks spreading false information. She questioned how long it would be before an AI-generated error caused real-world harm.

In the study, the BBC asked these chatbots to summarize 100 news stories from its website. During the review, the experts found that over 51 per cent of the answers provided by the chatbots had major issues. Additionally, over 19 per cent of the answers that cited BBC content had factual errors.

Some mistakes included:

-Gemini wrongly said the NHS does not recommend vaping as a way to quit smoking.

-ChatGPT and Copilot claimed Rishi Sunak and Nicola Sturgeon were still in office after they had left.

-Perplexity misquoted BBC News in a report on the Middle East.

BBC in its study found that Microsoft’s Copilot and Google’s Gemini had more errors than OpenAI’s ChatGPT and Perplexity. The chatbots often failed to separate facts from opinions and lacked important context.

Turness called on AI companies to stop summarizing news inaccurately. She wants them to work with publishers to find better solutions. Apple had earlier pulled back its AI news summaries after similar complaints from the BBC.

Pete Archer, BBC’s Programme Director for Generative AI, said that publishers should control how AI uses their content. He also urged AI companies to be more transparent about their mistakes and how they process news.

An OpenAI spokesperson said they are working to improve citation accuracy and respect publisher rights.