Anima Anandkumar on Foundation Models for Physics and Neural Operators
Caltech professor Anima Anandkumar explores the challenge of building foundation models for continuous physical systems such as weather, fusion, and fluid dynamics. She explains why traditional transformer scaling fails in scientific domains due to limited data and extreme context length requirements, requiring structural priors and inductive biases instead. Anandkumar details her work on Neural Operators and Fourier Neural Operators, which power models like FourCastNet to run accurate weather simulations on consumer GPUs. The discussion also covers applying Neural Operators to predict plasma disruptions in fusion far faster than classical methods. Additionally, Anandkumar covers TorchLean, a framework for writing and formally verifying neural networks in Lean, and her appointment to the United Nations Scientific Advisory Board.
Original sources
Podcast Latent Space
Newsletter Latent Space
AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.