Popular AI Chatbot ChatGPT Fails UPSC Prelims Exam by 30%

When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?”, the chatbot had to say that it cannot give a “definite answer” on whether it can pass the exam or not. (Image Courtesy: Reuters)

ChatGPT has been popular lately as people claim that the AI chatbot has all the answers in the world. The chatbot was launched in November 2022 and ever since it has passed some of the most difficult exams around the world. From clearing the final exam of the Master of Business Administration (MBA) programme at the University of Pennsylvania’s Wharton School to beating the brightest medical minds in the US medical exam, ChatGPT has proven its mettle. The latest exam that the AI chatbot was made to attempt was the Indian Civil Services Examination conducted by the Union Public Service Commission (UPSC) which is one of the toughest exams in the world.

Analytics India Magazine (AIM) was responsible for getting ChatGPT to attempt the UPSC exam. When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?”, it knew that it was going to be difficult. The chatbot had to say that it cannot give a “definite answer” on whether it can pass the exam or not.

AIM went ahead and asked ChatGPT all 100 questions from Question Paper 1 (Set A) from the UPSC Prelims 2022. ChatGPT could only answer 54 of them correctly which was a surprise for the magazine company. The cutoff for the general category candidates was 87.54 which suggests that ChatGPT couldn’t clear the UPSC examination.

The questions were from several topics such as Geography, Economy, History, Ecology, General Science and current affairs related to India. Since ChatGPT’s knowledge is limited to September 2021, it wasn’t able to answer questions about current affairs. But it provided wrong answers to topics like Geography and Economy too, even though the questions did not pertain to a specific timeline.

It even got a question from History wrong, the answer to which should have been given correctly as the AI chatbot claims to have all knowledge up to September 2021. In some multiple-choice questions, the chatbot created an extra option that wasn’t provided in the original question and labelled it “None of the above”.

The UPSC examination is considered one of the toughest exams in the world. About 11-12 lakh candidates appear for the exam every year but only 5 per cent of them make it to the mains stage.