Google DeepMind announces Genie 4, a world model with built-in sound

What is Genie 4?

Genie 4 is Google DeepMind's new world model. A world model simulates an environment: you set something in motion and it works out what happens next, how the world responds, how it sounds and how the people in it move.

DeepMind says Genie 4 builds on the Gemini family. Gemini Omni anchors the simulations in real-world appearance and behaviour, and other Gemini models help keep the simulation coherent.

What can you do with Genie 4?

DeepMind describes recordings of real-time interactions, where people use standard keyboard controls to move and steer the camera, and text prompts to change the scene as it runs.

With natural language you can change the weather, move the camera or introduce a new person or object, and Gemini adapts your prompt to the surrounding scene.

DeepMind also says Genie 4 aims to model physics such as aerodynamics and collisions.

Does Genie 4 make sound?

Yes. DeepMind calls Genie 4 its first real-time world model to generate audio natively. Each world has its own ambient sound, and characters can hold interactive dialogue.

Who can use Genie 4?

Not everyone yet. DeepMind has started rolling Genie 4 out as a research preview for trusted testers, to gather feedback first.

After that it plans to release it to consumers and developers, starting with Google AI Ultra subscribers in Project Genie and early access partners of the Gemini API and Google AI Studio. There is a waitlist for Project Genie.

What are the limits and safeguards?

DeepMind says visual details and object permanence can still shift or drift over time. It lists complex physical interactions, audio accuracy, legible text, spatial accuracy and long-term consistency as areas of active research.

It also says all video and audio generated by Genie 4 will include SynthID, a watermark for identifying AI-generated content.

Why does it matter?

DeepMind says understanding and simulating the world is a crucial part of building AGI. It also points to uses such as training environments for robots and agents, and new interactive experiences.

What's next?

Access widens in stages, starting with Google AI Ultra subscribers and early access partners. DeepMind has not said in the announcement when the general release will come.