AI Radar

Noteworthy AI conversations and writing, summarized from publisher notes.

OpenAI Jalapeño: Better Than Nvidia Blackwell

· SemiAnalysis

OpenAI has developed an inference chip named "Jalapeño," which was recently announced at Hot Chips. This chip, built in partnership with Broadcom, is designed exclusively for LLM inference and achieved a manufacturing tape-out in approximately 16 months. Despite being a first-generation chip, Jalapeño reportedly leads the industry, outperforming Nvidia, AMD, and Google chips on multiple top open-source models, a feat attributed to extreme hardware-software co-design. Contrary to some beliefs, Jalapeño is a generalized inference chip, not specialized for OpenAI models, and delivers high performance across various workloads. The chip's design prioritizes power efficiency, achieving superior token throughput per MW compared to competitors like Nvidia's Blackwell and Vera Rubin, even without speculative decoding. OpenAI's architectural choice of a homogenous chip pool, rather than disaggregating prefill and decode, aims for adaptability to changing workload mixes over time. The A0 stepping of Jalapeño shows impressive results, with a B0 stepping already in the fab promising further improvements in performance per watt.

AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.