ChatGPT Failed to Pass UPSC Prelims Exam : Reports

Mar 06, 2023, 16:45 IST

March 202 - In a recent activity, the highly performed and widely popular Artificial intelligence bot Chat gpt was tested with one of India’s toughest exams, UPSC Preliminary Exam 2022. The chatbot could only answer 54 questions out of 100 from question paper 1, Set A of the exam. It can’t even clear the 87.54% cut-off of 2021 General Category students.

The question paper was filled with various subjects such as geography, economics, history, ecology, science, current affairs, social development, and politics. Many people have used ChatGPT for various assignments and get huge popularity due to its ability to write work emails in specific tones, styles, and instructions. It can engage in conversations, answer questions, and provide information on various topics.

It can also analyze and interpret language patterns, understand contextual cues, and adapt to different styles of writing. Additionally, ChatGPT can generate creative and original responses, making it a versatile tool for language-related tasks such as translation, summarization, etc.  When asked about its capability to pass the UPSC prelims exam, ChatGPT failed to give a definite answer. It acknowledged that although The bot has a vast amount of knowledge and information on the UPSC examination and related topics. 

However, it pointed out that passing the exam requires not only knowledge but also critical thinking skills, application abilities, and time management skills.

A magazine went on to ask ChatGPT all 100 questions from the question paper. The chatbot struggled to answer some questions correctly, and in some cases, it provided wrong answers or created its own alternative answer choices. When they asked which of the following countries shares a border with Afghanistan and the options were Azerbaijan, Kyrgyzstan, Tajikistan, Turkmenistan and Uzbekistan, The bot answered all first four countries but the correct answer was Tajikistan, Turkmenistan and Uzbekistan. However, ChatGPT correctly identified only Tajikistan, Turkmenistan, and Uzbekistan as the countries that share a border with Afghanistan.

The Analytics India Magazine noted that ChatGPT's knowledge is limited to September 2021. It can't be able to answer questions on current events. However, the chatbot provided wrong answers for topics such as economy and geography, which are not necessarily time-specific.

ChatGPT may have impressed users with its ability to complete various tasks, but its performance in the UPSC Prelims exam shows that it has a long way to go before it can match human capabilities in certain areas. 

The performance of the AI chatbot ChatGPT in the UPSC Prelims exam raises important questions about the limitations of AI chatbots in tasks that require critical thinking and application abilities. While the chatbot has gained huge popularity in various fields, its inability to pass the UPSC exam highlights the need to recognize the limitations of AI technology. As AI chatbots continue to evolve, it is crucial to acknowledge their strengths and weaknesses and use them accordingly. To get the best outcomes in different areas, it's important to find a balance between using AI chatbots and human intelligence.