Google introduces Genie 2, its new large-scale AI foundation model for playable 3D environments and more

Google Genie 2

Google DeepMind has launched Genie 2, a foundation world model capable of generating 3D environments for training and evaluating embodied agents. This model uses a single prompt image to create action-controllable and playable virtual worlds, which can be navigated by human users or AI agents with keyboard and mouse inputs.

Capabilities
Genie 2 allows users to rapidly prototype and interact with diverse 3D environments. It builds on the functionality of its predecessor, Genie 1, which focused on 2D environments, by expanding to 3D worlds. Trained on a large-scale video dataset, it enables simulations involving character animation, object interactions, and environment physics. The model also supports counterfactual scenario generation, offering multiple trajectories from the same initial conditions.

Action Controls and memory
The model responds to input actions, such as movement commands, accurately associating them with the correct objects or characters. Genie 2 also retains long-term memory, rendering previously unseen portions of a world consistently when they come back into view. This functionality supports coherent interactions over extended simulations.

Environment generation
Genie 2 can generate and sustain consistent 3D environments for up to one minute. It creates scenes with varying perspectives, such as first-person views, isometric angles, and third-person driving visuals. The environments can range from forests and historical settings to urban spaces and extraterrestrial terrains.

Applications
The model is designed for training AI agents in richly simulated worlds, potentially enabling advancements in generalist AI. These worlds can also support creative workflows for prototyping interactive experiences. The model’s ability to simulate object affordances and interactions, such as opening doors or interacting with destructible objects, provides diverse training scenarios.

Development
Google DeepMind emphasized responsible development for Genie 2. The research builds on years of gaming-focused AI innovations, such as AlphaGo and AlphaStar, underscoring the importance of diverse training environments for advancing AI capabilities.