A Brain Too Big to Carry: On-Device vs. Datacenter Inference for Robots
The article examines the "embodiment problem" in robotics, where hardware constraints dictate model design, unlike LLMs. Key challenges include real-time control loops requiring low latency and the high upfront cost of on-robot compute. Frontier robot models are currently smaller than LLMs, with sizes determined by data availability and hardware fit rather than optimal quality. The piece discusses the potential for scaling laws in robotics, but notes data and network constraints. It details how different layers of a robot model (planning, action, servo) have varying frequency requirements, impacting which can be offloaded to a datacenter. The supply chain for silicon and memory also favors datacenter GPUs over dedicated robot chips, influencing the total cost of ownership and scalability of different inference approaches.
Original sources
Newsletter SemiAnalysis
Issue:A Brain Too Big to Carry — On-Device vs Datacenter Inference ↗
AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.