ChatGPT o3 Stuns with AIR 4-Level Score in JEE Advanced Mock Test
IIT Kharagpur Engineer’s Experiment Reveals Massive Leap in AI Problem-Solving
In a striking demonstration of AI progress, ChatGPT o3, the latest model from OpenAI, achieved a mock score in the JEE Advanced 2025 that would earn it a staggering All India Rank 4—a feat that has stunned educators, engineers, and tech enthusiasts alike.
Not Just a Test, But a Real Simulation
Anushka Aashvi, an engineer from IIT Kharagpur, conducted the experiment not as a gimmick, but under rigorous exam conditions:
- The AI was prompted to “act like a JEE aspirant”—no external tools, no web access, no memory from previous questions.
- Each question was posed in a separate chat session, preventing context carryover.
- No hints or corrections were provided—just raw, question-by-question evaluation.
Despite these constraints, ChatGPT o3 scored 327 out of 360, excelling across sections.
Where ChatGPT o3 Excelled
The model displayed extraordinary strength in:
- Mathematics and Chemistry, scoring a perfect 60 in both during the second phase.
- Step-by-step logical reasoning, even with multi-concept problems.
- Handling advanced calculus, organic chemistry, and inorganic reaction mechanisms with precision.
- Solving symbolic and abstract problems that require years of human preparation.
Anushka noted that the AI’s confidence and clarity resembled that of a top-tier aspirant.
Where It Struggled
Despite the impressive performance, the model still showed key limitations:
- Difficulty with graph-based interpretations.
- Inaccuracies with visual-spatial reasoning tools, such as Vernier scales.
- Delays and errors in solving image-based or tactile questions.
These flaws underscore a persistent gap in visual reasoning and real-world sensory understanding—areas still challenging for language models.
Why It Matters
The Joint Entrance Examination (JEE) Advanced is considered one of the toughest academic exams globally:
- Out of 15+ lakh aspirants, only ~17,000 qualify for the IITs.
- Questions combine multiple concepts, often leaving even top students uncertain post-exam.
- To see an AI outperform 99.999% of students is both groundbreaking and unsettling.
A Glimpse into the Future
In 2022, GPT-4 scored comparably low, with an estimated mock rank in the 10,000s. The leap to an AIR 4 equivalent score in less than three years highlights AI’s exponential growth.
As Anushka reflects:
“It’s humbling to watch an AI solve in seconds what we spent years preparing for.“
This moment may very well echo the historic 1997 Kasparov–Deep Blue match—only now, it’s happening in academia.
What’s Next?
The takeaway is no longer “Can AI compete with humans?”—it’s how soon it will surpass us across a widening range of disciplines. And as next-gen models approach, we may soon witness AI not just solving problems but setting entirely new standards of intelligence.
