Waymo’s Wild Ride: Navigating Tornadoes and Elephants

Imagine an autonomous vehicle cruising down a desolate highway when suddenly, a massive tornado looms in the distance. What does the driverless vehicle do next?

This scenario is just one of many that Waymo, a leader in autonomous vehicle technology, can simulate in its new “hyper realistic” virtual world. Collaborating with Google’s DeepMind, Waymo has developed the Waymo World Model using Genie 3, Google’s latest AI world model that can create virtual interactive spaces based on text or image prompts. However, Waymo emphasizes that Genie 3 goes beyond mere game development; it can construct photorealistic and interactive 3D environments tailored specifically for the challenges of autonomous driving.

Simulation plays a crucial role in the advancement of autonomous vehicle technology, allowing developers to test their vehicles in diverse scenarios without any physical risk to passengers or pedestrians. Through virtual environments, autonomous vehicle companies like Waymo can subject their vehicles to a wide range of tests, accumulating millions or even billions of simulated miles to better train their systems for any possible “edge case” they may encounter in real-world situations.

What exactly are these edge cases that Waymo is testing? Apart from the tornado scenario mentioned earlier, Waymo can simulate scenarios such as a snow-covered Golden Gate Bridge, a flooded suburban neighborhood with floating furniture, a fire-engulfed area, or even an unexpected encounter with a rogue elephant. In each simulation, Waymo’s robotaxi uses lidar sensors to create a detailed 3D map of the surroundings, including any obstacles on the road.

According to Waymo, the Waymo World Model can generate a wide range of scenes, from ordinary driving scenarios to rare and unusual situations, utilizing multiple sensor modalities. The company highlights three key features of Genie 3 that make it ideal for creating virtual worlds for their robotaxis: driving action control, scene layout control, and language control. These mechanisms allow developers to simulate various scenarios, customize road layouts, and adjust factors like time of day and weather conditions to mimic challenging real-world conditions.

Furthermore, the Waymo World Model can transform real-world dashcam footage into simulated environments, ensuring a high level of realism for virtual testing. It can also create longer simulated scenes that run at increased speeds without compromising image quality or computational performance.

By pushing the boundaries of simulation to simulate even the most improbable scenarios, Waymo aims to proactively prepare its autonomous driving system, the Waymo Driver, for rare and complex situations. This innovative approach underscores Waymo’s commitment to advancing the safety and reliability of autonomous vehicles through cutting-edge technology and rigorous testing.