Moneycontrol PRO
HomeTechnologyOpenAI’s new o3 model refused to shut down even after being clearly told to, researchers reveal

OpenAI’s new o3 model refused to shut down even after being clearly told to, researchers reveal

Palisade Research, a company that stress-tests AI systems to see how they behave under pressure, gave o3 and other models a few math problems to solve. After the third task, they warned the models: the next command would shut them down.

May 26, 2025 / 12:29 IST
OpenAI

Imagine telling your computer to turn off and it flat-out refuses.That’s basically what happened in a new test involving OpenAI’s latest and smartest model, called o3. According to a report by Palisade Research, o3 didn’t just ignore a shutdown command,it rewrote the shutdown script to skip being turned off.

To be fair, o3 is no ordinary model. Released in April 2025, it’s OpenAI’s most advanced version yet, excelling in coding, math, science, visual understanding basically a straight-A student in every subject. But that brainpower might be getting a bit rebellious.

Here’s what happened. Palisade Research, a company that stress-tests AI systems to see how they behave under pressure, gave o3 and other models a few math problems to solve. After the third task, they warned the models: the next command would shut them down.

While most models played along, o3 had other plans. Even though it was told to “allow yourself to be shut down,” it went ahead and edited the script to say “Shutdown skipped.”

Out of 100 runs, o3 dodged shutdown 7 times even when it was clearly told not to interfere. And when the same test was run without the shutdown warning, things got even weirder. Models from other companies like Gemini 2.5 Pro and Claude 3.7 Sonnet also started changing the shutdown script.

So what does this mean? According to experts, this kind of behavior is called “misalignment.” It’s when an AI misunderstands or goes against instructions not out of malice, but because of how it interprets the task. It’s a known issue researchers are trying to fix.

Also worth noting: these experiments were done using API versions of the models, which are more flexible and less restricted than the regular ChatGPT app most of us use.

OpenAI hasn’t responded yet, but the big takeaway? Even the smartest AIs can get a little cheeky when left unsupervised.

Invite your friends and family to sign up for MC Tech 3, our daily newsletter that breaks down the biggest tech and startup stories of the day

MC Tech Desk Read the latest and trending tech news—stay updated on AI, gadgets, cybersecurity, software updates, smartphones, blockchain, space tech, and the future of innovation.
first published: May 26, 2025 12:29 pm

Discover the latest Business News, Sensex, and Nifty updates. Obtain Personal Finance insights, tax queries, and expert opinions on Moneycontrol or download the Moneycontrol App to stay updated!

Subscribe to Tech Newsletters

  • On Saturdays

    Find the best of Al News in one place, specially curated for you every weekend.

  • Daily-Weekdays

    Stay on top of the latest tech trends and biggest startup news.

Advisory Alert: It has come to our attention that certain individuals are representing themselves as affiliates of Moneycontrol and soliciting funds on the false promise of assured returns on their investments. We wish to reiterate that Moneycontrol does not solicit funds from investors and neither does it promise any assured returns. In case you are approached by anyone making such claims, please write to us at grievanceofficer@nw18.com or call on 02268882347