Nvidia's Neuralangelo AI model can create 3D models from 2D videos

The model can create "lifelike virtual replicas" of buildings, sculptures and more.(Image: Nvidia)

Nvidia has announced a new AI model, Neuralangelo, that can create 3D "lifelike virtual replicas" from 2D video clips.

The company says that the AI model can generate "3D structures with intricate details and structures", and these models can then be imported into editing and design applications for further tweaking.

Neuralangelo works by analysing a 2D video clip of an object that has been filmed from various angles. It selects several frames of reference from the video to get an idea of what the object would look like from various angles. This allows it to get a sense of depth, size, and shape of the object.

Once analysed, the AI model then creates a rough 3D representation of the object, and then optimises it by adding details and textures. The resulting 3D model can then be used in, "virtual reality applications, digital twins or robotics development".

“The 3D reconstruction capabilities Neuralangelo offers will be a huge benefit to creators, helping them recreate the real world in the digital world,” said Ming-Yu Liu, senior director of research and co-author of the paper on the AI model.

“This tool will eventually enable developers to import detailed objects — whether small statues or massive buildings — into virtual environments for video games or industrial digital twins," added Liu.