Moneycontrol PRO
HomeNewsTrendsPopular AI Chatbot ChatGPT Fails UPSC Prelims Exam by 30%

Popular AI Chatbot ChatGPT Fails UPSC Prelims Exam by 30%

ChatGPT could only answer 54 questions correctly which was a surprise for the magazine company.

March 03, 2023 / 21:33 IST
When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?”, the chatbot had to say that it cannot give a “definite answer” on whether it can pass the exam or not. (Image Courtesy: Reuters)

When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?”, the chatbot had to say that it cannot give a “definite answer” on whether it can pass the exam or not. (Image Courtesy: Reuters)

ChatGPT has been popular lately as people claim that the AI chatbot has all the answers in the world. The chatbot was launched in November 2022 and ever since it has passed some of the most difficult exams around the world. From clearing the final exam of the Master of Business Administration (MBA) programme at the University of Pennsylvania’s Wharton School to beating the brightest medical minds in the US medical exam, ChatGPT has proven its mettle. The latest exam that the AI chatbot was made to attempt was the Indian Civil Services Examination conducted by the Union Public Service Commission (UPSC) which is one of the toughest exams in the world.

Analytics India Magazine (AIM) was responsible for getting ChatGPT to attempt the UPSC exam. When the chatbot was asked, “Do you think you can pass the prelim exam for UPSC?”, it knew that it was going to be difficult. The chatbot had to say that it cannot give a “definite answer” on whether it can pass the exam or not.

AIM went ahead and asked ChatGPT all 100 questions from Question Paper 1 (Set A) from the UPSC Prelims 2022. ChatGPT could only answer 54 of them correctly which was a surprise for the magazine company. The cutoff for the general category candidates was 87.54 which suggests that ChatGPT couldn’t clear the UPSC examination.

The questions were from several topics such as Geography, Economy, History, Ecology, General Science and current affairs related to India. Since ChatGPT’s knowledge is limited to September 2021, it wasn’t able to answer questions about current affairs. But it provided wrong answers to topics like Geography and Economy too, even though the questions did not pertain to a specific timeline.

It even got a question from History wrong, the answer to which should have been given correctly as the AI chatbot claims to have all knowledge up to September 2021. In some multiple-choice questions, the chatbot created an extra option that wasn’t provided in the original question and labelled it “None of the above”.

The UPSC examination is considered one of the toughest exams in the world. About 11-12 lakh candidates appear for the exam every year but only 5 per cent of them make it to the mains stage.

Invite your friends and family to sign up for MC Tech 3, our daily newsletter that breaks down the biggest tech and startup stories of the day

News18
first published: Mar 3, 2023 09:33 pm

Discover the latest Business News, Sensex, and Nifty updates. Obtain Personal Finance insights, tax queries, and expert opinions on Moneycontrol or download the Moneycontrol App to stay updated!

Subscribe to Tech Newsletters

  • On Saturdays

    Find the best of Al News in one place, specially curated for you every weekend.

  • Daily-Weekdays

    Stay on top of the latest tech trends and biggest startup news.

Advisory Alert: It has come to our attention that certain individuals are representing themselves as affiliates of Moneycontrol and soliciting funds on the false promise of assured returns on their investments. We wish to reiterate that Moneycontrol does not solicit funds from investors and neither does it promise any assured returns. In case you are approached by anyone making such claims, please write to us at grievanceofficer@nw18.com or call on 02268882347