Google DeepMind unveils SIMA 2, an AI agent built to think, plan and learn across virtual worlds

Google DeepMind has revealed SIMA 2, the newest version of its Scalable Instructable Multiworld Agent. It marks a significant step forward in training AI systems to reason through tasks, adapt to new environments and interact naturally with human instructions. The upgrade builds on the first SIMA model introduced in March 2024 and is powered by Google’s Gemini models, with an emphasis on planning and continual learning.

DeepMind says SIMA 2 can now analyse its actions and determine the steps needed to complete a given task. The agent receives a visual feed from a three-dimensional game world along with a user-defined objective such as “build a shelter” or “locate the red house”. It then breaks that goal into smaller actions and executes them using inputs similar to a keyboard and mouse. This approach allows the system to map instructions to meaningful behaviour based on what it observes on screen.

One of the standout advances is its improved performance in unfamiliar games. DeepMind tested SIMA 2 in environments it had never encountered before, including Minedojo, a research-focused adaptation of Minecraft, and ASKA, a Viking-themed survival game. In both cases, SIMA 2 outperformed the original version by demonstrating better adaptability and higher task success rates. The system also handles multimodal prompts, allowing users to give it instructions through sketches, emojis or different languages. Concepts learned in one game can transfer into another, enabling more efficient learning across varied virtual worlds.

Training the model involves a blend of human demonstrations and automatically generated annotations from the Gemini models. Whenever SIMA 2 picks up a new skill or movement in a fresh environment, that experience is recorded and fed back into the training process. DeepMind says this reduces the amount of human-labelled data required and allows the agent to refine its abilities as it explores new scenarios.