NVIDIA's Ming-Yu Liu on World Foundation Models and Cosmos
Ming-Yu Liu, VP of Research at NVIDIA and IEEE Fellow, joins the NVIDIA AI Podcast to explain world foundation models and NVIDIA's Cosmos platform, which CEO Jensen Huang announced at CES. Liu describes world foundation models as deep learning-based space-time digital simulators that predict future states of the physical world — distinct from LLMs (which generate text) and from generic video foundation models (which prioritize creative use cases). World models are specifically focused on physics-consistent simulation and can be customized for different physical AI setups, such as varying numbers and placements of cameras on a robot or vehicle. Liu outlines three primary use cases for world foundation models in physical AI: (1) verification of trained policy checkpoints by simulating them across many environments before real-world deployment, (2) pre-training policy models so they require less real-world data, and (3) test-time planning, where a robot simulates multiple futures before choosing an action. The Cosmos development platform bundles pretrained world foundation models in both diffusion and autoregressive variants, video tokenizers, post-training/fine-tuning scripts, and a video curation toolkit. The models are released as open weights for commercial use, with the explicit goal of letting physical AI builders focus on robotics problems rather than training world models from scratch. Liu notes that diffusion models tend to produce more temporally coherent outputs, while autoregressive models — benefiting from years of GPT-style optimization — run faster and integrate more naturally into physical AI pipelines. NVIDIA is partnering with humanoid robot companies (1X, Figure) and self-driving firms (Wayve, Xpeng) to iterate on Cosmos. Liu acknowledges the field is still in its infancy: robust physics simulation and proper benchmarks remain open problems, and integrating world models effectively into physical AI systems will be a multi-year effort.
Original sources
Podcast NVIDIA AI Podcast
Episode:How World Foundation Models Will Advance Physical AI With NVIDIA’s Ming-Yu Liu - Ep. 240 ↗
AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.