AI Radar

Noteworthy AI conversations and writing, summarized from publisher notes.

OpenAI on Deep Research: End-to-End RL as the Path to Capable Agents

· Training Data

Isa Fulford and Josh Tobin from OpenAI's Deep Research team join Training Data to discuss the product launched three weeks prior—an agent in ChatGPT that synthesizes web sources into detailed 5–30 minute reports, positioned as OpenAI's second agent release after Operator. They explain that Deep Research is a fine-tuned version of o3 trained end-to-end with reinforcement learning on hard browsing and reasoning tasks, with access to a browser tool and a Python tool. The hosts press them on how it works under the hood, and the team emphasizes that the model's flexibility comes from end-to-end training rather than from a hand-authored graph of LLM calls—the model reacts to live web content and adjusts its own strategy mid-task, which they argue is hard to replicate with scripted agent frameworks. A central argument of the conversation is that the dominant lesson of machine learning—"you get what you optimize for"—now favors end-to-end RL tuning on top of pretrained reasoning models. Tobin uses Jan LeCun's cake analogy to explain why RL is "so back": pretrained language models are the cake and supervised fine-tuning is the frosting, so RL cherries can finally work. Fulford adds that hand-written logic should only encode hard rules (e.g., data the model must not touch), while everything else is better learned. They argue this recipe—state-of-the-art reasoning model + human-equivalent tools + direct optimization for the outcome—will scale to increasingly complex tasks, framing AGI as increasingly an operational rather than research problem. The discussion also covers use cases (knowledge work, medical research, shopping, travel, coding, and personalized education), design choices like the upfront clarification flow, the importance of high-quality training data, future plans to expand into private data sources and fuse browsing/computer-use capabilities, and reactions to Sam Altman's claim that Deep Research will take over a single-digit percentage of economically valuable tasks. In a lightning round they predict agents will be the breakout application category of the year and reaffirm reinforcement learning's resurgence.

AI Radar wrote this summary from the publisher’s notes with AI, not from a transcript. How it works.