‘Superintelligence’ could change everything, OpenAI warns it might also destroy everything; here's why

OpenAI warns that 'superintelligent' AI systems could pose catastrophic risks even as Microsoft, Meta, and others accelerate their race to build the next generation of artificial intelligence.

Artificial intelligence is advancing faster than many imaginedm and even its creators are starting to worry. OpenAI, the company behind ChatGPT, has issued one of its starkest warnings yet, cautioning that the next phase of AI evolution, known as 'superintelligence,' could pose catastrophic risks if left unchecked.

In a detailed blog post, the Silicon Valley company argued that AI’s pace of progress has already outstripped most public understanding. While ordinary users still see AI as a productivity tool, powering chatbots, code assistants, or image generators, OpenAI said its systems are already capable of outperforming humans in “challenging intellectual competitions.”

“The potential upsides are enormous,” the company wrote, “but we treat the risks of superintelligent systems as potentially catastrophic.”

The firm is now urging the creation of new global safeguards for advanced AI systems, frameworks that could mirror building codes or cybersecurity standards. OpenAI’s call to action includes a proposal for an “AI resilience ecosystem” and a shared safety principle among frontier AI labs.

“No superintelligent systems should ever be deployed without robust methods to align and control them,” OpenAI warned, calling for collaboration across governments, researchers, and industry.

Even as it raised red flags, the company acknowledged the transformative potential of AI. “AI systems will help people understand their health, accelerate progress in fields like materials science, drug development, and climate modelling, and expand access to personalised education,” the post read.

The timing of OpenAI’s note is striking. It comes as the company reportedly gears up for a stock market debut, and just as competitors like Microsoft, Meta, and Anthropic double down on their own 'superintelligence' projects.

Microsoft, which already has a landmark partnership with OpenAI, has now unveiled its own effort to build what it calls “humanist superintelligence.”

Mustafa Suleyman, the co-founder of DeepMind and Inflection AI who joined Microsoft in March 2024 to lead its consumer AI division, announced the creation of the MAI Superintelligence Team on Thursday. As per Fortune, Suleyman said the group will pursue “incredibly advanced AI capabilities that always work for, in service of, people and humanity.”

Suleyman’s framing is a deliberate contrast to OpenAI’s more technical vision. “We reject narratives about a race to AGI,” he wrote in a blog post. “We’re in this for the long haul to deliver tangible, specific, safe benefits for billions of people.”

Until now, Microsoft’s contract with OpenAI had capped how powerful a model it could build, limiting the company’s ability to train large-scale systems. “For a company of our scale, that’s a big limitation,” Suleyman told Fortune.

That restriction is now easing. The new MAI Superintelligence team, led by Suleyman and Chief Scientist Karén Simonyan, will operate as part of Microsoft’s broader AI division. The company has reportedly been investing heavily in AI chips to power these systems and poaching researchers from Google, DeepMind, Meta, and Anthropic.

Suleyman said the goal is to make Microsoft 'self-sufficient' in AI while still extending its partnership with OpenAI through 2030. “We have a best-of-both environment,” he told Fortune, “where we’re free to pursue our own superintelligence and also work closely with them.”

Microsoft isn’t alone. Meta rebranded its AI arm as Meta Superintelligence Labs in June 2025, while OpenAI co-founder Ilya Sutskever launched a startup called Safe Superintelligence, dedicated to building, and containing, such systems.

AI company Anthropic has also set up a division to research how to control a hypothetical 'superintelligent' AI before it exists.

So far, though, these efforts are more ambition than reality. No true superintelligence exists, and experts remain divided over whether it’s achievable using current methods. Still, the branding has taken hold, signaling both a new frontier and a new marketing race among Big Tech giants.