Microsoft introduces new gen AI model ‘Muse’: What you need to know

WHAM Demonstrator

Microsoft has introduced Muse, its first generative AI model for game ideation. Muse is a gen AI model of a video game “that can generate game visuals, controller actions, or both”. Before renaming it as Muse, Microsoft originally referred to the gen AI model as the World and Human Action Model (WHAM).

Muse is developed by the Microsoft Research Game Intelligence and Teachable AI Experiences (Tai X) teams in collaboration with Xbox Games Studios’ Ninja Theory.

Microsoft says Muse was developed around the idea of how innovations in AI capabilities can support creativity. The tech giant found the gaming industry to be the perfect use case for Muse.

The Redmond-based tech giant said in a post on the website of the scientific journal Nature that although gen AI is being used to generate text, images, audio, music, video, and gameplay sequences of video games, it often falls “short of the expectations of creatives, raising key challenges in integrating these technologies more fully into creative practices.” 3D game development deals with complex tasks like generating 3D worlds and their mechanics, creating in-game challenges, exploring interactions with NPCs (non-player characters), etc.

As per Microsoft (via the journal ‘Nature’): “Our WHAM models the dynamics of a modern video game over time. WHAM was trained on human gameplay data to predict game visuals (‘frames’) and players’ controller actions (‘Model architecture and data’ section). The resulting model accurately captures the 3D structure of the game environment (‘Model evaluation’ section), the effects of controller actions and the temporal structure of the game. The model can be prompted to generate coherent game situations, demonstrating consistency and diversity and the ability to persist some user modifications”.