What is Genie 4, Google’s new world model that can simulate humans, physics, and sound?

Google DeepMind has introduced Genie 4, its cutting-edge world model, capable of creating highly realistic visuals and interactive environments that users can alter by inputting text prompts during the simulation. This model possesses the ability to generate real-time audio that harmonizes with the simulated world, encompassing music, ambient sounds, and dynamic dialogue.

Introduced in a research preview on October 27, Genie 4 represents a significant leap forward in AI-generated virtual worlds, following its predecessor Genie 3, which was launched a year prior. The unveiling of Genie 4 places Google at the forefront of the rapidly expanding world model market, alongside competitors like Runway's GWM Worlds 2 and emerging entities like World Labs and Decart.

Genie 4's key features include generating highly photorealistic visuals and native audio generation, allowing users to make real-time edits by typing prompts to modify the weather, adjust camera angles, or introduce new elements. The model demonstrates improved physics and can sustain a consistent scene for up to two minutes, a notable enhancement over its predecessor.

The technology's most striking advancement is its ability to simulate human movements and speech, enabling real-time interaction with AI-generated characters that adapt to user inputs. This capability holds promise for various applications, such as educational experiences, film prototyping, and robotics.

Genie 4 relies on Google's AI models, including Gemini Omni and Gemini reasoning models, to ensure accurate and coherent simulations. The model is currently accessible to a limited group of trusted testers, with plans to extend availability to AI Ultra subscribers and early access partners in the near future. Google has implemented strict safety measures to prevent the generation of harmful or misleading content, and all generated content will be marked with SynthID, a digital watermarking technology.

Despite its impressive capabilities, Genie 4 still exhibits some limitations, such as potential inconsistencies in visual details and object permanence over extended simulation periods.