Who Grades the AI Models? | Ben Horowitz & Rayan Krishnan
The a16z Show features Erik Torenberg, Ben Horowitz, and Jennifer Li in conversation with Rayan Krishnan, founder and CEO of Vals. The discussion centers on the challenge of evaluating AI models as public benchmarks become saturated and models optimize for tests. Krishnan advocates for independent, continuously evolving evaluations, explaining why self-reported model scores can be misleading. They detail how VALS evaluates models before release and the complexities of testing increasingly agentic systems over extended periods. The conversation also covers the importance of evaluations for enterprises assessing AI ROI, the potential for token spend to rival employee salaries, and how evaluations could provide a shared language for various AI applications, from model routing to policy.
Original sources
Podcast The a16z Show
Episode:Who Grades the AI Models? | Ben Horowitz & Rayan Krishnan ↗
AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.