OpenAI's Greg Brockman on Compute, AGI Progress, and Human Attention Bottlenecks

Source material

Transcript

So Greg, thank you for coming back here.

I don't think we ever charge you for rent.

So maybe I'll set you an invoice later, but Greg, you've been part of two really spectacular companies.

Stripe as in play number four and then the first DTO.

I just recently heard that they process 1.6 billion.

Sorry, 1.6% of the global GDP.

You must be proud of that.

That's amazing.

You must be even more proud of the fact that OpenAI has almost the billion or maybe more than the billion in weekly active users at this point.

It's all very exciting.

It shows you what technology can do.

And you're not just co-founder and president, but you're also chief builder.

At OpenAI, I heard that that was one of your titles.

I'm not sure it's ever an official title, but I've been called many things.

Let's just say that.

Well, you have an audience of great builders here.

So we'll start from all the way at the bottom of the stack.

OpenAI has multiple stacks of the business, one of which is compute.

And you guys have been very aggressive, very aggressive on securing compute.

Why is that?

Well, in many ways, we have a very simple business.

We buy, rent, build, compute, and we resell it at a margin.

That's it.

As long as the margin is positive, then you want to scale it.

Because the demand for solving problems, the demand for intelligence.

That's unlimited.

And the AI's that we have right now really are able to rise to the challenge of effectively any kind of problem that you want to throw at them.

Do you have enough compute?

No.

Really?

Yeah, definitely.

Well, I was just with Matt Garman, and he says the GPU compute availability in 2026 rounds to zero.

Don't you guys have all of it?

I mean, we have, we would love more.

We're constantly out there hunting for more, honestly.

And I'll tell you, like when we first launched, when we launched ChatDBT, I remember being on a call with my team, and they were like, all right, how much compute should we buy?

And I said, all of it.

And they're like, no, no, seriously.

Like, come on, how much we buy?

And like, no matter how fast we try to ramp compute, I guarantee we're not going to be able to keep up with the demand.

And that has been true ever since.

That's fascinating.

Moving up from compute, since I don't know if much of this audience can help you with securing more compute.

Because most of them are founders of startups.

About architecture and scaling laws.

All right, what are the, what are the, where are we in the scaling laws?

Are they still doubling each year?

Are you changing architecture?

What, what is, what do you guys pushing on the frontier on the research side?

Well, I would say first of all, the scaling laws are a deep, very beautiful mystery, right?

They feel deeply fundamental.

It's like the scientific truth that just like you think about physics and, you know, Newton's laws and things like that.

There's somehow this truth of the universe.

And they're empirical.

Like we don't necessarily have all the theory to explain exactly why it works.

But to me, the most beautiful thing is that neural networks were really designed, like, in the 1940s.

But before they were computers.

And somehow, we've been able to take the exact ideas that were developed back then.

And apply increasing amounts of computation.

And as you pour more compute into the models, they get corresponding and more capable.

And it just keeps going.

There's no wall.

And I think that's a beautiful thing.

That's pretty beautiful.

Are there more research or more algorithms that are in the works?

Because, you know, in the past, we had neural networks to your point in 1940s.

But we couldn't, we didn't have the compute for it.

Now that we have the compute for it, you just, or are we just pushing the same things?

Or are there new architectures and new ideas coming up?

Yes.

So I would think of it as we absolutely have new ideas that are constantly powering what we do.

It's very simplified to say, well, let's take a neural network from the 1940s and, you know, put it in a gigawatt data center.

Right?

We have made tons of innovations.

And we constantly are improving things.

And sometimes these are micro tweaks.

Like you just realize that the wave and formatting data was not quite right.

And that can actually be a very big deal.

Sometimes it's larger.

You think about the shift from the LSTM to the transformer.

And I don't think the transformer is, you know, like, everyone's moved past the transformer as described in the other 2018 paper.

So there's, there's constant innovation happening.

And I think of places that have been, perhaps the most invested in long-term research on how to improve the architectures, how to improve the fundamental algorithms and how to get the paradigm shifts.

I think OpenAI has been leading the pack there.

And that's something we continue to invest and I see lots of fruit on the horizon.

And on the models, there's OpenAI have a formal definition for AGI.

We close and we not close.

Pat and Sonya published this thing that we are at AGI functionally.

Do you agree with that?

Do you not agree with that?

Well, we do have a formal definition.

But to some extent, one thing I have learned is that everyone has their own intuitions about what AGI is.

And maybe you can view it as, like, according to my view of where we are.

I think we're about 80% of the way there.

And that we have models that are smart.

They're very capable.

They're able to, if you give, are they smarter than you?

I mean, they're certainly more capable than I am at writing software, right?

If you give it all the context, then yes, I think that they are, they're just so capable.

It's really remarkable.

Is anyone here feel better at writing software than GPD5.4?

Oh, no!

All right, writing kernels.

So even there, we're seeing massive gains from, exactly, and for some of our internal results, that there, we're really seeing, if you pour the right kinds of, you know, if you have the right set up for your problem, then you're able to get really massive results out of very low level, even low level tasks.

And just to give you one example of how things have been trending, one of my system's engineers also, very similar it was like, hey, I haven't been able to get value out of the models of GPD5 or 5.1 for 5.2 as well.

For 5.3, on a large, prepared this design document for a very complicated system's optimization used about to do, he handed it over to the model, went to sleep, waking up and tending to like, give this to his team to work on for the next week.

And when he woke up, it was done.

That the model had actually implemented the initial spec, had seen that it was slow, had added instrumentation, had actually run the code, used to profile it, it figured out where things were slow, and iterated multiple times until it got into an optimized result.

And like, like, that is incredible.

That's where we are.

And so how, what would you advise all startups here to do?

Because the models keep getting more and more capable, they kind of, I've asked this, when Sam was here in the past, and you know, what, if you're building today, do you need to rebuild in two years when a new model comes out?

Because all the functionality and all the capabilities, all change around you.

Do you need to make sure that you're not in open AI's way because you're kind of wrong.

You're just going to run over startups because the models are so much more capable.

How would you recommend a set of startup founders to build in this environment?

Well, first of all, I would say to Lean in, the tools right now have become incredibly useful.

And if you look even over the course of December, I think that we went from these agented coding tools being like, you know, they're like writing 20% of your code, to writing 80% of your code, which means they go from being kind of a side-show to being the main thing that you're doing.

And I think we're doing that across all of the work that people do with computers.

All computer work this year.

And you can look at the recent progress on Codex.

It's really changing from a tool for software engineers, to a tool for anyone who's doing work with a computer.

And just over the past week, we've released a bunch of features.

They just make it so much more powerful and capable.

And the one thing we just announced today is a new tool called Chronicle that plugs into the Codex.

Where it actually can see everything you're doing with your computer and can form memories of what's going on.

And so you ask it a question, you just instantly knows what you're talking about.

You're like, huh, what was I doing five minutes ago at Nose?

You're like, oh, what was this person talking about at Nose?

To me, it was this real wake-up call to realize you spend so much of your effort right now.

Just explain these to your computer.

What's going on?

Like, why are you explain to your computer?

What's going on?

It makes no sense.

And so I think what's going to happen over upcoming years is the models are going to get much more capable.

What better harnesses will be able to be able to solve harder and harder problems.

Come up with the new knowledge, all of these things.

But there is a one-time shift that's happening now, which is really about context.

It's really about is your AI able to, you have all these meetings.

You didn't include the AI.

You know, that's not very nice to the AI.

Like, you're asking it to help you with things and it has no information.

So I think really leaning into how do you make sure the AI even has enough information in theory to solve the problem.

And then trust the models are going to really get there and improve.

So I think it will be a constant cycle of improvement and iteration and leaning into the tools and kind of talking to your friends if you're out there using it.

But that there is this investment, that's a one-time investment that now is the time to make.

And in terms of, like, let's say we, you set that a lot, but how is OpenAI using Codex differently than you think everybody else outside this using it?

Well, I think one of the amazing things about being at OpenAI is you do get to go in the future, right?

You do get to really see the shape of what's emerging and we can code design, right?

We can really change the models, the harness, everything together in order to better serve the needs that we see.

And a lot of the approach we've been taking started with software engineering.

And we set some clear guidelines for example saying that we still want a human to be accountable for all code that gets merged.

Right?

So at the end of the day, is it a good thing to merge this piece of code?

Is it well structured?

Is it going to make our code base more maintainable?

Want to make sure that a human who is signing off to say yes?

And that's, I think that thoughtfulness of not just saying, okay, it's just blindly used this or, you know, oh, we don't want to use this at all.

I think neither extreme is quite right.

And then we are also going vertical by vertical with an open AI to adopt these tools within finance, within sales, within IT.

And there we have this small dedicated team who's really deeply understanding the domain working with the people who are the experts in it in order to build skills in order to modify the codex, UI, whatever it is that is needed or to get it to be good.

And then that's something we can then once we have it in good shape, we will externalize and we're able to ship that to all of you.

And so we are starting to work with certain customers as well.

So for people who want to be very AI forward and want to be part of defining this revolution that there is a place for that.

And I'd love to talk afterwards.

But yeah, I think that just this desire to say, hey, we really want to be AI forward really good in the future and experience what it will be like for everyone else one year or two years, three years down the low.

Throw.

Do you guys structure a company differently or the engineering team differently because of the living in the future?

I mean, if you have to go way back when my father learned computer science, he was just himself and then we had these long software releases that became waterfall.

And then when the web happened and the cloud happened, we had these two pizza teams and we had to scrum.

Now that we have these coding agents how do you structure around everything differently?

I think we're still figuring it out.

And there's certain places where you really see it.

For example, the cost of building a prototype is cheap now.

It's so cheap.

And if you want to build a dashboard that used to be like, ah, take like someone like a week to do it and you just do it now.

And so actually a lot of the bottleneck has shifted to things like sharing.

Like how do you, and so we actually have some internal work on this as well that again, we will be externalizing of how do you make it really easy for anyone in your enterprise to build a dashboard a widget, a bot, whatever the thing is and then share it with others.

And then that starts to really put pressure on having good governance.

Like you want your IT organization to see all these different, you know, threads of execution that are happening, all the little things that are being shared around have some control over data provenance, right, to really make sure that, okay, like a good example of this is, I think people are now starting to take their internal knowledge and external development to wikis.

We have some really cool one of these internally.

And the thing you immediately think about as well if someone has a document in the internal knowledge base that was accidentally permissioned incorrectly and they realized, oh no, I didn't want this information to be accessible.

How do they fix that, right?

So normally it's they go into the dock, they change the permissions, but now there's these derived artifacts.

And so you need to make sure you have some way of tracking through the system to say, well, this output document came from this source one, the source one is no longer accessible to this audience.

Let's go and invalidate that as well.

And so you have to start really building your technical architecture with awareness of the way that people are going to use this information and it really changes how teams relate to each other because you can just, it really changes where the bottlenecks are and what's hard.

Do you think teams like this are going to be able a lot smaller or we're going to have still human software engineers in a decade?

Well, a decade is a long time from now and that this ceiling on this technology is hard to, is really hard to internalize.

I think that it is clear that what a company is will change in a lot of ways.

I think that we're going to have this ability for solar printers to build very incredible businesses and so anyone who has a vision I think will be able to realize it.

I think the jobs that you all have will become way easier in a lot of ways, way more fun.

Now, might be more competitive too because everyone's going to have these amazing tools and it's really figuring out what is your niche, what is your unique angle is probably going to become the most important core.

But a lot of how we've run organizations right now and it's almost only one way to organize large groups of people where you have teams, you have management structures and you have these hierarchies and all these things.

Maybe that can change.

Maybe you can be much more flat, small teams that can really just do incredible things.

Like we're seeing it right now in mathematics where these individuals in the internet are using GPD-54 Pro to solve these unsolved math problems.

And we need a math team and they're just doing it.

Yeah.

My son's a math nerd.

I just told him that maybe we should be studying something else besides math.

But I, well, let's see, this is the question, if you look at something like AlphaGo, you know, move 37, this move that just like changed humanities understanding of the game.

But the thing that was surprising is it made the game more interesting and important for humans.

And maybe that'll be true for these other domains too.

True.

What about common failure modes when you're building your building with production, agenic workflows?

What do you see as the common things that founders get wrong and they're building incorrectly?

Well, I think that these models they have such power and really understanding how to operate them well takes thought.

And so we've been investing a lot in primitives, security primitives, observability, having, again, good governance, things like that.

But just to give you one anecdote that I think is evocative.

I asked, so I was working with my code to ask the students to install some package that someone had open-eyed written around two in error.

I was like, oh, ping that person on slack and ask them for help.

So ping the person on slack.

Two minutes later, it's like, this is taking too long.

I've escalated to the person's manager and actually pinged to the person's manager.

And you realize it's like, on the one hand, it's kind of a reasonable thing for the model to do.

It's being proactive.

It's trying to solve my problem.

It's like, you know, not just sitting around waiting to be told what to do.

But on the other hand, like, you know, maybe you should have taken a little bit longer.

Maybe you should have checked with me.

And so I think that really thinking about these questions where we're still building up the EQ of the model.

And that in some places, it's getting very good.

For example, cooking a prove a prove a prove is kind of where we've been.

And humans are not very good at that either, right?

They just, they just default.

They just default.

And so now we're starting to have a eyes that can actually take care of flagging.

Is this a high-risk action?

Hey, this one should be escalated.

This one's okay to auto-approve.

And it really makes you realize that human attention is going to be this incredibly scarce resource.

Right?

The doing of things now is easy.

The, is this a good thing?

Is this what I wanted?

Is this aligned with my values?

My desires?

That is going to become the single most important bottleneck.

And so I think building systems that take that into account and really think about the human factor.

Like that's the most important thing to do now.

Another human factor is security.

How would you advise people to think about security in this world of AI?

And sort of about breaches left and right with Versailles recently.

And then these models are incredibly powerful at finding security halls.

So how would you recommend people here use the models to find those security issues?

Well, I think there's a couple of levels to the answer.

I do think that this is, I think that the internet has been a place where security has been just like a of ratcheting, important concern over time.

You think about where it started going through the 90s with viruses and worms and malware and those things.

And we've moved past that.

I think we are also moving now to a much more secure regime.

But it does require kind of an internet wide effort to get there.

And so a lot of this honestly is just again leaning into the technology, having these models they can scan your code base, they can actually be used for end-to-end red teaming.

Like there's a lot that can be done with them.

And a lot of how we're thinking about further models and improvements there is really leaning into how do we actually sort of leverage trusted access programs, how do we leverage the community of people who really care about being defenders and making the internet more secure.

And I think that's something where everyone has a role to play and can participate.

But the number one thing is just sort of recognizing that these models are very powerful.

But they're not magic, right?

That they are just like a part of the overall resilience ecosystem.

And I think that we as a society and I think every company, again, really contributes to this, have something to build in terms of how do we incorporate these in a way that results in more assurance and more sort of certainty on the impacts of whether it's a particular patch that you're taking, whether it's thinking about how do you make sure that you're just sort of rolling in updates quickly as they're being released.

So I think that there's a lot of work to be done but I have a lot of optimism for where this is going.

Let's switch to speed.

Seems like things are moving faster and faster and faster when the world of accelerating change we were talking about it when we, when you're walking up here around how you're trying to keep up with things.

How do you keep up with all the accelerating change?

How would you recommend everybody here keep up with everything that's changing?

Well, I think this is the new normal and I think to some extent it's not really because of AI.

I think it's just been the trend of technology for the past two decades.

There's more people doing things.

It's easier to do things than ever.

The better to enter it goes down.

It means it's also much more easy to build value to have great successes.

And so I think that really trying to keep your ear to the ground and understand what's changing.

And to some extent it always starts with the same thing, which is play with the technology yourself.

It's very different to hear AI described versus to use it.

But the beautiful thing about AI is it's so intuitive.

That's the whole point is that rather than have the machine be something you have to contort yourself to, the machine contorts itself to you.

It's doing work for you and it should be something where you ask it and does something.

And so I think that just really trying to just get your finger on the pulse of what's changing, what's possible, where the model's lag.

That is I think the core skill that is going to really determine a lot of the success of companies in the future.

And then on the flip side of that, you guys have held back models to work with security agents.

So it's like the opposite of going as fast as possible.

So you're doing things responsibly too.

So how do you think about the balance?

Because you're in a competitive environment.

You want to ship as quickly as possible.

And yet you're trying to do the right thing as well.

Yeah.

I think at a values level, like what open AI is about?

Like we really want to put the power of AI in people's hands.

Like we believe that people can, we want to empower people to build the future with the tools that are being created.

But we need to do that in a thoughtful way, that we really think about both sides of care of the benefits.

Here's the risks.

How do you maximize the benefits?

How do you mitigate those risks?

And I think that in cybersecurity and bio-security those are areas where we're very thoughtful.

We've been building.

We've been working on these kinds of both mitigations and trusted access programs for quite a long time.

And that what we see coming is models that are going to be increasingly powerful and capable in a continuous way across all dimensions of capability.

And we announced last week the expansion of our trusted access for cyber program.

Are there anyone here applied?

No one.

Oh, I see one hand, two hands.

Okay, more of you should apply.

It's great.

We really need help because it's very important that people who are trustworthy and responsible and really want to push these models are participating in this because that is how that's going to pay dividends for everyone.

We're going to have more to announce over upcoming weeks on how we're expanding the program.

But, and also when we release models to everyone kind of the mitigations that we have and how we're going to tune those to be both to really balance.

To really try to bring these capabilities broadly as possible while also making sure that the ones that are thinking about the risks and able to have some observability over them and to ensure that this is maximum positive in terms of deployment.

So I think the short answer is like, it's quarter our mission.

We care a lot about the impacts of what we're doing, not just building the technology and isolation, but it is a whole community, a whole world effort to really get to where we need to be.

Moving up from the models to the application layer, which is what a lot of people here are building.

How does open AI decide what in the application layer you're going to build and what you're going to leave out?

Well, people have probably seen the word focused being applied to open AI quite a lot recently, especially for the first time in a while.

And it's been applied to her too.

And it's hard because the field of AI is one of opportunity, right?

It's like anything you're going, you can imagine.

It's going to be great.

No question is going to be great.

And as a company, as a single company, no matter how much compute we build, no matter how many people we have, are only going to be able to do so much.

And so a lot of where we've been thinking about things is what is the most focused strategy that covers the parts of the space.

You know, maybe it's an 80-20 or just like the parts of the space that we think we can have most impact on.

And I think there it's very clear right now we're going through this agentic transition.

And so products that are, and it's not just about enterprise versus consumer, right?

So it's like clear we are being very serious about enterprise, like we're selling to big companies and building a whole muscle and sales motion there.

But consumer, what consumer is going to change, right?

It's kind of a very broad term that buckets and multiple things.

But the slice of consumer that's about not just productivity, but about goals, about achieving your goal, about even knowing what is your goal, being able to list it that and having an AI that can practically do that.

It's all kind of the same thing.

Like in the end, we're trying to build an AGI that you can talk to that has all this context that you can use in your personal life, your work life.

It's trustworthy, right?

You can go to it for advice and give you useful information, maybe health information or maybe about finances or, you know, about if you're trying to figure out what to do with your career.

Like all these things, they all kind of will not are into one thing.

And it's meant we had to make some very painful decisions about what not to do.

But I think I would just say that that's the aperture that we look at things through and that things that accrue to that singular vision of what we want to build, you should expect us to pursue.

I think we'll be coding with command lines and agents in a few years or it's going to be completely changed.

I mean, I think that we're in a very unnatural state right now for how we work.

Like we all sit behind this box and kind of type away.

And it's very clear our bodies were not designed for this.

We got our carpal tunnel and our, you know, hunched shoulders and all these things.

And I don't think we want that.

I don't think any of us wanted that.

Like I think that we wanted more free time.

But it's not yet about free time necessarily.

Right?

It's like you want to spend more time with your loved ones.

Yes.

You want to spend more time talking to people and like coming up with like brilliant visions or just like what you're excited about or just understanding yourself.

So it's kind of like, do you want to be a CEO of an organization of like 100,000 agents?

Like that actually seems pretty good.

And I think that we're all going to be able to get so much more done but the mechanics of it are going to feel as different as like going from having to write out things with, you know, by hand with a quill or something to being able to, you know, just send a text message and have people going and, you know, working on your behalf on your goals.

All right.

We talked about compute.

We talked about model and security and agents and app layer.

It's not about frontier.

When, when are the models going to be good enough to push the frontiers of science, physical AI?

Seems like we had Jen found here.

Seems like LMS had been a great scaling law for digital intelligence.

It hasn't been as strong for robotics or physical intelligence for aspects of biology and science where the problems are probably a lot harder to verify or takes a long time to verify.

Well, how are you keeping track of science and physical AI in the world?

Well, science is one domain that we're really leaning into and we see line of sight to really incredible progress.

And we're starting to have some signs of life and I think it's always important to ground in what is happening today when trying to predict what will happen six months a year from now.

So, for example, we had a physics result where our AI came up with this very beautiful formula that physics has been working on this for quite some time.

So, it's probably impossible, though it's maybe an unsolvable problem.

And it's pretty significant, right?

It's like real serious physicists who really do this as a step towards really being able to get to some sort of answer for quantum gravity and all these things, not there, but it's a step.

That's much bigger than where we were just a couple months ago.

And so, it makes you really wonder a year from now, like, how far will we have traveled?

Now, things like biology that they are different from physics and math, right?

That they are, you got to leave your beautiful simulated world and deal with messy reality.

But I think we've been learning how to deal with messy reality and other domains, software engineering is a perfect example where we've really realized that just building the thing that solves competition, programming competitions, like that's not enough.

Like, you need something that's seen real world messy code human's interrupting in different ways, like, this ad versatile banging at it.

And so, I think that on science, I expect we're going to see a real Renaissance.

You know, maybe we'll see some big results this year.

Next year, I think it's going to be a totally wild, wild time.

We live in interesting times.

I promise that I get you out on time because you're a busy man.

Before we let you leave, we got one minute on the shot clock.

What, since you have no time, but soon you will have lots of time.

What do you and Anna do for fun?

Fun.

I mean, the same as anyone, like to watch movies, go on hikes, those kinds of things.

You know, not as much time for it as, as maybe we'll hopefully have post-AGI, but you got to kind of enjoy the ride along the way.

Thank you, Greg, for joining us.

Thank you, everyone.

Thank you.