Nvidia CTO Michael Kagan: Scaling Beyond Moore's Law to Million-GPU Clusters
Michael Kagan, CTO of Nvidia and former co-founder of Mellanox, makes the case that the $7B acquisition of Mellanox was the key unlock turning Nvidia into the dominant AI compute platform. He argues we have moved past Moore's Law, where performance doubled roughly every two years via transistor density, to an AI-driven regime where model sizes double every three months, demanding roughly 10x performance growth per year. Nvidia meets this by treating the data center as a single unit of computing, scaling up via NVLink within rack-sized "GPU" systems of 72 chips, and scaling out via Mellanox-derived networking (InfiniBand, Spectrum-X Ethernet, BlueField DPUs) to connect 100,000-plus GPUs—and eventually millions—into one fabric. Kagan details the hard problems at this scale: with millions of components, hardware failures are statistically certain and must be absorbed in software; network jitter, not raw bandwidth, is the real bottleneck because applications hide communication behind computation; and geographically distributed data centers introduce speed-of-light latency variance that requires endpoints—not edge buffers—to manage congestion. He also pushes back on the assumption that inference is lighter than training, noting that generative AI and reasoning models (which run many sequential inferences per prompt) make inference compute demand comparable to or greater than training. Nvidia is responding with pre-fill- and decode-optimized GPU SKUs sharing a common programming model via CUDA. On the Nvidia–Intel partnership, Kagan frames it as a "win-win" expansion of the pie rather than head-to-head competition: accelerated computing and general-purpose x86 are complementary, and pairing them opens new market channels. Looking forward, he sees optical interconnects, Earth-2-style simulation as experimental science, and AI helping humans discover new laws of physics as the next frontiers. His "Huang plus Kagan's law" targets sustained order-of-magnitude yearly performance gains, supported by Nvidia's accelerated annual product cadence.
Original sources
Podcast Training Data
Episode:Nvidia CTO Michael Kagan: Scaling Beyond Moore's Law to Million-GPU Clusters ↗
YouTube Training Data
Video:Nvidia CTO Michael Kagan: Scaling Beyond Moore's Law to Million-GPU Clusters ↗
AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.