OpenAI's Nick Turley on ChatGPT's Path to a Super Assistant

Source material

Full source text

Chat GPT originally was entirely free, and the reason for that was that it was intended to be a demo.

- Yeah.

- And we were going to wind it down after a month.

We then realized that the demo went viral when people loved the demo.

And it was actually a product, but we realized that to be a product, you can't take the product down every time you're at capacity.

So we shipped subscriptions simply because it could shape the demand.

It was a way of gracefully turning users away, and we had to turn away someone.

You guys are at 900-mVTE active users now, and that grows.

This has been incredible, the next billion users.

Where are they going to come from?

We've got about 10% of the world coming to us now, 90% left to go.

There's so much more opportunity.

Well, Nick, so excited to have you here.

Thank you for having me, Puru.

You've had quite the journey, from Germany to the US, from Brown.

That's true.

Most recently, at Instacart, delivering groceries in 30 minutes, you're now delivering AGI to billions.

I'm sure that was a plan all along.

Yeah, clearly, total master plan.

Well, tell us about your journey.

How did you get to OpenAI?

I know it's a fun story.

And your three and a half years or so at OpenAI, how have they gone?

The only through-light in how many sort of employment decisions have been entirely people-based.

So I don't claim any credit for joining OpenAI.

We're predicting chat GPT or anything like it.

But I hit up someone who I admire a lot, who I got to know at Dropbox, Joanne, who worked here at the time.

And I asked her to get off the DALI 2 wait list.

And she told me I had an interview if I wanted to get off the wait list.

So I took the bait and got totally nerd sniped in the process.

And here I am.

There you go.

The DALI 2 wait list will get you.

Great recruiting tool.

Nice.

We should do more wait lists, probably.

Yeah.

Well, you know, the big super cycle we're in is chat GPT.

Now, I assume over a billion users on the monthly side, 900 million weekly active users that recently reported up from zero, three and a half years ago.

You could have, if I imagine what the dashboard of Nick Turley looks like, it could have users, it could have paying subscribers, it could have daily active users, it could have retention engagement.

I mean, there's like 15 things, maybe all of them.

What is your North Star?

How do you, how do you, what are you optimizing for?

What is Nick looking at in his daily dashboard?

It's funny, right, because it's such a young product.

It's been to your point three and a half years.

And this kind of question, it kind of changes as you all when you grow up and you ask yourself, you know, what are we really building here?

And to this day, I want to build a super system that can actually help people achieve their goals.

And ultimately, the thing we care about is like, is our product doing that?

Is it actually helping you do the thing that you're coming to the product to do?

And it's so different for different people, right?

Some people are trying to get healthy.

Other people are trying to start a company, learn a new topic, do their taxes.

There's all the different things that you might be doing.

And the true measure of success is whether or not we're helping you do that.

And obviously, we look at WAU in particular because, you know, we want to know if you're coming back to the product.

We look at retention.

But we, you know, we look at all kinds of stuff in aggregate because really there isn't like this one single thing that you can optimize for.

If you were to allocate 100 units of points to these metrics, which metric can you distribute the 100 units across these metrics in order of importance for you right this second?

It's a good question.

I care a lot about long term retention.

And I would put all my points there because I'm really proud of the retention stats we have.

But ultimately, the sign of durable values, whether or not people are coming back in three months because that means you're really solving their problems.

And I think things like revenue, they follow from that versus trying to go on those things directly.

And we've had a lot of success making very principled decisions on this stuff.

Like one good example is in GPT-4 used to be behind a paywall because we didn't serve it to everyone.

And then we had GPT-4, which was a total breakthrough in our ability to inference it.

And so we just gave it away for free.

And that ended up being totally revenue positive and retention positive because it just provided access to the tech.

And I think when you make your decisions that way and you focus on the customer, you end up with a great product and revenue obviously follows too.

Phenomenal.

Well, it shows up in the numbers.

You know, I posted this chart yesterday on the data that we have from a third party, the retention curve for chat GPT or smiling.

Look at that just like that.

And that is a very rare occurrence, as we know.

And why do you think like if you were to give us a narrative on that smile curve, what is the why do these smile curves exist?

What are you seeing in chat GPT that has people who have maybe turned off for a couple of weeks or months coming back?

And why are they coming back?

Look, there isn't one single thing.

The way that you build a retentive product is lots and lots of little things and really trying to make it better systematically.

I will say that with AI and in particular chat GPT, I found that it takes people some time to really understand all the parts of their life they can delegate.

And I think many users for that it's a multi-month process for them to understand how can this thing help me and what are all the different ways that I can plug chat GPT into my life.

And but when I think about some of the breakthroughs and levers we've had things like search and personalization, they have helped solve those user problems because search provides way more daily value to you.

It used to be that chat GPT was a pretty worky product.

We'd say it used to go down on the weekend.

We'd use it go down during the summer months when a lot of people were off from work.

And today we're mobile first.

The vast majority of us are mobile.

And we see all these personal use cases.

And I think search was a big investment that got us there.

And personalization makes chat GPT so much more relevant for you.

Right.

Because it gets to know you over time.

You get to know it.

And those are two things that have materially moved the way that people come back to the product.

But there's lots more to do.

And as mentioned, I'm not resting on our retention stats, even though we're obviously very proud.

Nice.

Nice.

Nice.

And you know, the other thing that I got wrong about chat GPT was this is two and a half years ago.

I was like, well, you know, let's look at who's going to win this consumer race.

You typically these consumer markets are winner take most winner take all.

Look at search.

Google has near 90 percent plus market share.

Three and a half, four trillion market cap mobile.

Same thing with Apple.

Social.

Same thing with meta.

As a well, AI meta has all the distribution.

Google's got all the distribution.

They've got three, four billion users.

Well, it'll be a flick of a switch for them to roll out their AI.

But I was wrong.

That's not what happened.

Chat GPT turns out, you know, you guys are at 900 million VTE active users now.

And that growth has been incredible.

Clearly, distribution was not enough.

Right.

So the same question for distribution.

What are the levers for us that have gotten us to the scale?

Is it model quality?

Is it product quality?

Is it features?

Is it the experience or product improvements like memory and personalization or search?

Like same question, like what would you say drove historical growth and success?

We've got about 10 percent of the world coming to us now.

90 percent left to go.

Right.

There is so much more opportunity to reach more people and introduce them to the way that I can benefit them.

Right.

But when I look backwards and I only say that because like the next billion users might be very different in terms of like how you engage and reach and provide value.

But when I look back, it's been roughly sort of one third, one third, one third between sort of classic friction removal type of work.

Like one of the biggest moments when you look at pure impact was removing the authentication wall.

And Sam will say I told you so because I think that was his feedback from like day one.

It's like you can't you shouldn't have to log into chat GPT.

But it's like stuff like that that you do for any product.

And it does matter.

Some things never change.

Right.

But then another third or so is our what I would sort of sort of core product investments.

And they're really typically things that we've done together between research and product.

So search and personalization are really good examples of that where we came together and we figured out not just you.

You ex evolution, but also how to post train these changes into the model.

And it was really the moments when we came together.

Another recent example is like we have these writing blocks that render when you ask about queries where you're trying to write with the model.

And like putting really good craft into those experiences really matters.

And our users love it.

And then another third of the growth has been just model improvements like step, both step changes like going from GPT 3.5 back then the GPT 4, then going from GPT 4 behind a paywall to 4.0 suddenly available to everyone.

Right.

But a lot of it is also the iteration that isn't splashy that doesn't warrant like a name to release.

And we I'm really excited about the updates we just made with 5.3 5.4, etc.

Because that is when we take a lot of user feedback and we methodically address it.

And obviously that shows up in our retention as well.

So sort of one third, one third, one third between classic friction removal and access core product investments and then pure model improvements.

And so the question that I've really been waiting to ask you is how do we get the next billion?

And and talk about that a little bit.

There is a lot of it seems like at least from the outside fog of war.

If I was a consumer today in the market to pick my super assistant, you'd have a couple of great options, you know, Claude out there.

They're having some great traction last couple of weeks.

Gemini, mega distribution, Uber distribution and us by the by the by the leading product today, at least in user numbers.

The next billion users.

Where are they going to come from?

First of all, just to contextualize that goal, we care about two things at the end of the day.

Obviously, reaching more people is really important.

Reads the direct manifestation of our mission to the world where the more people we can introduce the benefits of the eye, the better that is.

But we're also really excited to go deeper.

And that means taking the same billion users that find value in chat today and actually providing more meaningful value in the world, like actually helping them achieve their goals, not just answering questions.

Right.

So I'll talk about how we get to more skill, but I think it's important to remember that the way this technology is evolving is, you know, we're going to go beyond pure chat bots pretty fast.

I think on on skill, it's shocked me how many people have found value in chat as it works today, because I don't think delegation is a natural skill for most.

And chat is a pretty.

It's a it's a power tool, right?

You come to it, it doesn't tell you what it's for.

You kind of have to discover it on your own and you have to use it and then you'll learn about this prompt that was really cool.

And then maybe you you're on Twitter and you read about another one or you're on Instagram and you learn another one.

But the product, it's like a raw appliance.

And I think for two on one thing we really need to nail as we reach the next set of users is a product that has a bit more of an affordance.

Because I think for most people, they're very, very busy.

And they everyone, I think, in the world has intelligence constrained problems, problems that more intelligence could help with.

But you need to frame that to people.

I still feel like we're a little bit too much like a computer terminal and it needs to feel more like software, like a new or an operating system of software.

So that's one thing.

Another thing that gets at the same constraint is beginning to be proactive.

In a world where a lot of folks are too busy to delegate their problems to AI or don't quite know where to start, I think being able to help you proactively is really, really important as well.

But, you know, I think all of these are product evolutions that we could make on top of the current tech.

And the thing that gets me particularly excited is productizing our next generation tech or reasoning models.

Because the truth is, when you look at reasoning and chat to be due today, it's relevant to a very small group of people.

It's relevant for the people who are trying to get the most out of chat to be to.

But I fundamentally believe that reasoning, it's transformative.

And if you can figure out how to productize reasoning in a way that works on people's behalf without them even knowing.

And that looks very much like the model doing long, long, long tasks on your behalf.

It doesn't mean you encounter the concept.

It just means it's benefiting you.

Right.

So there's so much work to do.

And, you know, the product certainly has to evolve to be relevant for this kind of scale.

Yeah.

One of the one of the things that I've been hoping for a while and, you know, Brad made a bet two years ago.

When can chat GPD help me take actions?

Can chat GPD help me be more proactive?

And I think his bet expired end of last year.

So he's very curious.

When is that coming out?

I'll frame that for you because, you know, with with with search engines and Google, you know, two decades ago, you got on the 10 blue links, you spent an hour getting the answer.

You can now get the answer instantly with strategy.

And it feels like the next step is is actions.

100 percent.

And it feels like the next step is, you know, with, you know, pulse is a great proactive product that that feels.

You know, I have a pulse that runs weekly.

But what I would really like is like, hey, Nick spoke about something and just just find me.

Make sure I know that Nick spoke about this or like, hey, this X, Y, Z thing happened that I cared about a lot.

When is when is that going to get proactive?

What is the modality going to look like?

Yeah.

So there's two concepts, I think.

There's chat GPD doing stuff rather than just answering and then it's chat GPD being proactive.

And I think when you put them together, you start feeling like it feels like a super assistant because I think these things compound.

On the action taking piece, strictly speaking, chat GPD can do stuff today.

The action space is just very limited.

Right.

It can search the Web, which means it can use search tool or browser in the same way that a human would.

It can make images.

It can do all these things.

Right.

But it doesn't.

It clearly doesn't have the same action space that a human with a computer would have.

And that is what we have to build.

And if you timing is everything on these bets.

Right.

And I don't pretend to be great at timing either.

But I look at past attempts that we've made, like the chat GPD agent, for example, which kind of has capabilities like this.

It was just slightly too early.

The models weren't quite good enough to hit real escape velocity.

And the problem is, if you don't have escape velocity, is that users don't learn to trust it.

They don't even try.

So when you look at a lot of things people were doing in the original version of chat GPD agent, it was the things that happened to work, like migrating your file server into the cloud or something like that.

Useful stuff, but very niche.

And as this stuff gets better, we just have to get it to a point where people try to use it for real meaningful problems in their life, because then we can start hill climbing.

And this has been the magic of chat GPT, where chat GPT upon launch was good enough to get real attempts at use cases, even if they didn't initially work.

Chat GPT was a pretty bad writer originally.

It was a bad software engineer.

But people tried and got enough value out of it that we could take those use cases and make them great.

And I do think we're about to get to that point with general purpose agents where it works well enough that you get at least partial credit.

And because you're getting partial credit, you get really good tasks back.

And then the magic begins because once you have a set of use cases that you can climb the hill on, we can make them awesome.

So on tasks, I think we're close.

But I think even people inside of OpenAI would have had a hard time predicting exactly when this gets good.

We've been excited about it for a while.

On proactivity.

Pulse was a really great first step because we wanted to build was a form factor where you're not prompting the model like the models prompting you.

For the reasons that I described earlier, which is it's so hard for people to delegate to figure out what their problems are.

What if the understood your your goals and things you're interested in and just could start being proactive on your behalf?

Pulse is limited in the value you can provide for you because it's not connected to your life and it can't take action.

So it's producing information for you.

And people love that.

I love that.

I've got mine running, too.

But I think the magic begins when you have actions and productivity because then it can begin speculatively actually detecting.

Hey, you just landed where you were supposed to go.

I'm going to call a cab for you.

Or if you're at work, it's like, hey, I proactively ran this analysis because I saw your metrics dropped.

So I think these things really compound and we need to nail multiple of the building blocks to really achieve the transformation of the form factor that we hope for.

As you were answering those questions, I now have 15 more questions for you.

So I hope you have 15 more minutes.

But OK, one by one, we'll start with what you said on actions and tasks.

Got it on timing.

Tough to say.

But is there a shape or ordinality of tasks and or agents that that you think, hey, this is the kind of thing that's likely to come first whenever it does?

I mean, the thing that's already come first is the domain specific agents.

Right.

If you look at what's happening in in code, we're fully there.

It's mind bending.

But we've got so many engineers who don't open their ID like ever.

And for me, as someone who used to code and then unfortunately got very, very busy, it's brought me back in the game.

So Codex and products like it is clearly a product that has escape velocity where people are absolutely using it for all kinds of a genteck work.

And if you just take what people are doing and make it work even better, you kind of get all the way there.

I won't be surprised if you see this happen for other forms of sort of quantitative knowledge work just because it happens to have the properties that code has.

It's testable.

You know if it worked or not.

It's very R.L.

friendly.

But the domain specific ones already work.

I didn't think everyone's working for is general purpose agents that just kind of work for anything.

And that's why I think you need to win a consumer because it's very hard to train people into like, OK, it can work.

Deep research was a consumer product.

It really was our first genteck thing out there.

But I think what consumers want is I can just ask it anything and we'll do what needs to be done without any sort of retraining.

We'll get there just a matter of time.

At least a psychological goal is flight bookings.

Totally.

Restaurant bookings, shopping, all this stuff.

There are so many consumer problems and those are just the type of things that you would kick off.

The minute you have productivity, there's there's things you don't even think of as a genteck tasks like you're trying to get in shape.

You don't think of that as a task you would delegate unless you have a trainer.

In which case you do.

But most people don't.

Right.

But the I knew that it could certainly start working in the background for you over very long periods of time and getting you.

You know, here's your fitness plan.

OK.

I actually send you up for this thing.

You could imagine it being quite helpful if it's aligned with your with your long term.

You're going to give a zen picker on for the money.

We got to be careful what businesses we get into.

But hopefully we can help.

That'll be great.

Cannot wait.

Cannot wait.

The second thing you said was, you know, proactive users and that might require us to go beyond chat bots.

What's an example of a modality that might take tragedy beyond a chat bot?

So chat will always be close to my heart.

It's the way we grew up.

And it's an important modality.

I think it's less about chat and more about natural language to me, where the fact that you can express yourself to the machine in ways that are very natural to you, whether that's text, whether that's voice, whether or not that is, you know, structured UI that is rendered by the model.

That is just very, very powerful.

And that's here to stay.

But I think the thing as a server.

That's right.

For those that don't know, that's the name of our code base short for super assistant server, because it's proof that this was always the vision and it's always the vision.

But the thing that will change, I think, is that chat is a great way of expressing your intent.

It's a good way of communicating with the machine.

But it's not a great output.

Where in many cases, what you want back is an artifact.

Here's your plan for your trip.

Here is the analysis.

Here is an outcome that I delivered for you.

I just made you five bucks.

Like, this is what I want my AI doing for me.

Yeah, totally.

I mean, this is what people care about.

And I think chat will always be there as the way that you sort of disambiguate your intent and you kick off the task.

But I don't think it's necessarily the final deliverable.

And I think that's the way in which we can evolve.

So hopefully that's a very graceful transition, because I'm very lucky and it's hard earned to have a billion people coming to you weekly for a thing that they love.

But I think it's a great jumping off point because we have so much unsatisfied intent from people where they're trying to clearly trying to do something and chat is helpful enough.

But it could be so much more helpful.

I think that's where we evolve.

Yeah.

And you must be sitting on so much of this data where people are showing up to chat, GPD and attempting, as you said, three years ago, they were at least making the attempt.

Yeah.

So you might have at least a frequency histogram of like, hey, here are all the things that people want to achieve with us.

We do.

We have like really awesome classifiers that run automatically.

It's fully privacy preserving, but gives us a sense of what use cases people have.

And it's important, right?

Because when you make a new model, we can model update, you want to know what use cases just got better, what cases got worse.

And that's not always always trivial to figure out unless you have really good analytics on the system.

But so much of my learning is actually qualitative, where I will just have a habit of reaching out to a fairly random set of users and just figure out what they're doing.

And I've never worked on a product where three and a half years later, you're still learning every time, because usually by that time, you know what the use cases are that your product can deliver on.

But our tech is so unusual in the fact that I keep learning about something crazy I didn't know was possible.

Wow.

That's awesome.

But basically, a billion users, I suspect a small fraction of them are power users who are getting maybe thousands, maybe tens of thousands of value on their $200 subscription.

The vast majority is middle of the back.

And then a few call it casual users who start using chat GPD as search maybe, or teach me about AI or help me with my homework.

What is your focus?

Maybe in those constituents, power users, casual users and early users or how server you frame it, what is our focus on for each of those three factions?

Yeah.

Yeah.

Well, first of all, I feel accountable to our entire user base.

In fact, our non users too, because products like chat GPD can have real externalities on all humans.

But when I think about sort of the way we build, it's really useful to imagine the extremes.

One extreme being a user who doesn't care about AI at all, who has a busy life and needs to be convinced of the value that we can provide.

Because that forces you to really nail the interface and to expose the capabilities that are hidden in the model in a way that people can actually grok.

And then the other useful extreme is our power user base, because power users are the users who teach us what's possible.

It's actually impossible for us to do all the product discovery on our own simply because of how empirical this technology is and how much you actually learn post launch.

So building for each of those extremes can be valuable.

But our user base is incredibly diverse and people have so many different use cases.

And this is why I like to look at all kinds of different segmentations, not just frequency, but also what use cases are you coming to us for.

But definitely huge variety in the chat GPD user base.

I look up to macOS, for example, as an example, where it really works for people who don't understand technology at all.

It's entirely magical.

But if you are a power user, you've got terminal, you've got settings, you can configure almost anything in macOS.

And it's really beautifully done where the complexity is progressively disclosed.

So you can interact with it and love the simplicity of it all.

But you can also get all the knobs and developers love it.

Right.

And so I think this is kind of the inspiration for how we want to be in chat GPD.

That doesn't mean we always live up to it, but it means that building for power users is extremely important.

And that's not just a property that I think is sort of aesthetically exciting.

It's also really important in the eye because it's the power users who show you what's possible.

They are actually doing the product discovery because it would be impossible for us with such an empirical tech to do all the product discovery on our own.

So the type of user who subscribes to chat GPD Pro, who used Codex before it quite worked, who is now the strongest advocate of tools like Toget's and teaching us what's what's possible.

That is an incredible, incredible, incredibly valuable member of the community.

And it might not show up in your weekly active users.

It's just one number.

Right.

But this is exactly why there isn't like a single North Star.

And you really need to need to need to take these different segments very seriously.

So I love building for power users.

And you know, you asked on, you know, token consumption, et cetera.

It's so fascinating to see there's people who get incredible value out of these products and watching what they do is very informative.

OK, so so we're very focused on the entire user base.

Learn a lot from the power users.

You know, the other thing I might say is the power users right now are getting a lot of value, almost too much value and a lot of no such thing.

No such thing.

The the analog that is most common is the Uber and Lyft of the 2015 era.

Right.

And it took it took a while, but I know you were thinking about it a lot.

I know you guys are thinking about pricing quite a bit.

Yeah.

Maybe tell us a little bit about pricing.

You know, right now, pricing is pretty simple.

Is there is there a path for for for folks who are getting a lot of great value to price that product differently and meet them where they are?

And the other way on the other side and pricing is there's no world in which pricing doesn't significantly evolve when the technology is changing this quickly.

Right.

Chat, GPT originally was entirely free.

And the reason for that was that it was intended to be a demo.

And we were going to wind it down after a month.

We then realized that the demo went viral and people loved the demo.

And it was actually a product.

And we realized to be a product, you can't take the product down every time you're at capacity.

So we shipped subscriptions simply because it could shape the demand.

It was a way of gracefully turning users away when we had to turn away someone.

And it felt like the fairest and most equitable way of doing so is saying, hey, you know, if you really need this product, pay a subscription fee and you got it.

Then we figured out to make the product stable and we had the choice of do we keep the subscription thing or do we go back to free?

And we realized we had consistently more tech that we couldn't scale.

GPT for being the first example because we had way too many free users to serve GPT for and we put it behind the plus plan.

And so, you know, the way we stumbled into subscriptions was was sort of accidental by trying to just solve for the user.

And it felt like the right way at the time to to provide maximal access to our to to to our tech.

Since then, we've had so many other breakthroughs, including test time compute, where you can scale up intelligence kind of as much as you want, more or less.

And it took us in the entire industry a little bit of time to turn that into product value.

But we're here now where our our our power users want to use more and more and more intelligence.

And it's possible that in the current era, having unlimited plan is like having unlimited electricity plan.

It just doesn't make sense because people may need a lot, a lot of electricity and they're getting a lot of value out of that.

There's a reason you can't buy that.

Right.

So obviously, I want to be really thoughtful about the way that we evolve our our plans and schemes and subscriptions.

But I'd be incredibly surprised if it didn't change, given the magnitude and profoundness of the technical breakthroughs that we've had and the product breakthroughs that follow.

Yeah.

And you know, relatedly, so I imagine you're going to have something for the power users.

What about the other side?

How do we get the casual users in through the into the wheel and still monetize them?

As mentioned, you know, our business model will evolve in the North Star's access.

We would like to pick a way of providing an offer.

We want to provide an offering that maximizes the number of people who can who can access our most powerful tools.

I think for the longest time, that has been subscriptions.

Subscriptions are the downside of the fact that, you know, in many markets, people don't have credit cards or they don't use credit cards to subscribe to software.

And we're interested in other ways that can maximize access of the tech.

Our ads pilots are in that spirit.

We really view it as a tool of bringing chat to B.T.

and our intelligence most broadly to anyone around the world.

And it is an example of how we constantly need to evolve and figure out the best way to to bring the demand in line with what we are able to offer.

Makes sense.

Makes sense.

You know, the ads the ad species is has been a tricky one because, you know, Sam has historically expressed reluctance about ads and, you know, you've got to maintain a lot of trust with while delivering that.

So I guess what changed?

I think we've talked about this several times in my mind straight open and every time it came up, we said if if we were to do ads, we'd have to be really thoughtful.

But we do it.

So the first thing we did, you know, starting, you know, end of last year was to really engage the company on if we put ads in chat to B.T., how should we approach it?

What should the principles way?

What be how do you preserve the things that are magical about chat to B.T.

while getting the benefits of ads, which is our ability to bring our most advanced tech to anyone regardless of their ability to pay.

And I really love where we ended up on the principle side on the experience side.

We're very, very early.

But on the principle side, I feel really proud because it's very important that the answer of chat to be to be independent as an example.

Respecting user privacy is very important and there's a lot to learn from the way that tech has evolved over the last few years or the last decade.

And I like that the principles are out there before we've even really gotten started.

Like we're very early with our pilots.

Yeah, it's kind of interesting.

It was obviously very anxiously and eagerly looking at our support in bounds and data.

And the most common inquiry about ads is not how do I disable ads or turn off ads, but it's like, how do I run an ad?

Because the entire ecosystem is really excited to be part of the story and to figure out a way to talk to chat to B.T.

users.

So there's a lot more to come.

But I'm very eager to get this right.

Yeah.

Yeah, I'm sure you guys will.

Searching gears, Nick, something you and I have spoken about a little bit is distribution and partnerships.

There was a couple of big partnerships last year.

Apple Reliance with Gemini.

Those are two big user bases, right?

A lot of India, a lot of the iOS users.

Doc, tell us a little bit about how you think about partnerships for chat GPT to meet the user base and maybe specifically on those two as well.

Look, I think partnerships are a great way to bring two products together and to expose something like chat GPT to people who might not otherwise have encountered it.

The thing that I care about most when considering something like a partnership is what is the user experience and can we make it amazing?

Because at the end of the day, when you look at what's going on in the market, you can get users to click on things, you get them to tap any sort of product, especially if it looks like a product they recognize, etc.

But if the experience isn't truly awesome, people will churn or they will at least not retain in the way that we've been lucky to retain them on chat GPT.

For that reason, I'm super interested in paths like that, but it needs to be great.

It's the accruity of the user.

We are very lucky to have a great brand and a recognizable product for many folks.

And I want to make sure that anything we do is accretive to all that.

Nick, you are a master of tradeoffs.

You must be making a lot of tradeoffs right now.

Tell us about some of the tradeoffs you're making.

Tell us about a tradeoff that you might be making that people don't appreciate from the outside.

There are a lot of tradeoffs indeed for different reasons.

What I encounter a lot is trading off, delivering on the use cases that exist in the product today and making them better, versus productizing step change technology that's going to generate a whole other set of use cases.

Because when you think about how chat GPT came to be, it was a totally open-ended product.

It was basically a user experience around a technical breakthrough.

We couldn't have told you all the ways that people find it valuable, but putting it out there was really important because it allowed us to discover in the world what you can do.

And then post chat GPT, we can obviously very systematically go and improve on the things that people actually want to use it for.

And when you're at a company in this moment where you both have such amazing traction with what exists today and the most mind-bending breakthroughs on the research side, the balance you have to strike is making the core product you have better today with all the things that matter, latency, reliability, making the use cases really great that people come to with providing access to the step change.

And we try to get the balance right, but we're a small team and we don't always get it right.

And for that reason, that's one of the most difficult trade-offs that I have to do it.

Nick, I imagine one of the hardest trade-offs you guys make here is those GPUs that are melting between chat GPT, between codecs, research.

How do you guys allocate the GPUs?

That is a very good question and I'll let you know when I figure it out.

Just kidding.

We've gotten a lot better at this.

I really hope, by the way, to be at a point one day and I've yet to reach that point where we don't have to face this trade-off because it's really painful to have real user demand for products that you can't serve.

If you've only ever worked in software, that's an entirely unusual dynamic, where you are limited by this zero-sum resource out there.

The marketplaces have it, but I think pure software doesn't really have the dynamic right.

So one thing we try to do, obviously, we prioritize our existing users first.

We want to provide a fast, reliable product and that is critical and table stakes.

Then when you look at new capabilities, the sort of naive business school thing to do would be to probably look at incremental revenue per GPU or something like that.

But this is where it's more an art than a science because we often have new breakthrough capabilities that are entirely zero to one.

Deep research was one of those.

We couldn't have told you, "Is there going to be demand for a consumer demand for a research product?"

But if you don't productize it to find out, you will never know.

So this is where we have to be a little bit thoughtful on how we balance things that are no-brainers, that people are really going to love with things that are brand new ideas.

Then obviously on the research side, there's a reason that Mark has the job he has because a big part of his job is figuring out what research to fund.

Obviously, GPU is a big part of that.

So very nuanced topic that we're continuously getting better at.

But for me, the priority is always on our users.

The other takeaway that I had is you don't have line of sight to a time when you won't have that problem.

It's been so fascinating because we obviously have been incredibly lucky to encounter more and more users who want to use our technology.

But then the value that we're able to provide for each user is going up as well.

And GPU consumption correlates pretty well with that value.

And when you just look at token consumption per user, especially in the enterprise too, which is a massive opportunity, you see a lot of very GPU-hungry workflows and demand keeps going up even as prices go down.

This is a fascinating insight.

People are used to saying that humans, you can't kind of make more humans.

Well, it takes nine months and then ten years.

But you're saying that's actually more and less finite resource than GPUs.

Yeah.

I mean, on the human side, you can hire more humans.

And obviously, we've been busy doing that and bringing the best talent across functions to OpenAI.

In the world with agents, you can also get more leverage per human.

You can make your humans very effective at their job to do more.

But GPUs are zero-sum.

And if you don't have more GPUs, you really have to figure out how do you make very, very hard trades and hate making hard trades for users.

And it's a desire to have more GPUs.

But it's useful to start with the most zero-sum trade-off when you do your planning.

So I think starting working backwards from GPUs is usually a pretty good idea.

Yeah.

You know, we have all these external data sources for charts of users and usage and activity and retention and all those things, but we don't have a token per user over time.

And I bet that chart is like a sweet line going this way.

I think internal is pretty good.

Our internal employees is a pretty good indicator for what's about to happen.

And yes, the charts are mind-boggling.

Yeah.

Yeah.

Fascinating.

OK.

A couple of quick ones on the present before we go into the landscape, which is, you know, shopping.

You know, we just moved into a new house.

We took some photos and we were hoping that all our furniture would magically appear, that Chat GPT helped us paint.

But, you know, a lot of recent updates on Chat GPT shopping.

Tell us about it.

What are you thinking on shopping as Chat GPT as a shopping assistant?

Shopping is one of those use cases that exist organically in Chat GPT today, and they work.

You can ask Chat GPT about any purchase you might be planning and get pretty excellent advice.

But it's also one of those cases where the experience that exists in Chat today, it's not the perfect experience that you would want, because shopping is very visual, for example.

So you're going to want to actually see products and images and be able to compare and contrast, read walls of text.

People care about the sources of, you know, where can I learn more about a given product, etc.

And so there's a lot of work to do to make this discovery really, really good and allowing people to use Chat GPT as an assistant to find the right product to buy.

And that's where our focus lies, making that really great and making that really great in a way that works for our retail partners as well.

Because as I mentioned earlier, it is huge appetite from the ecosystem to be part of the Chat GPT journey.

And nailing the discovery piece has been the most promising focus here today.

Niko, in Chat GPT, you must see a breadth of information.

You must see a breadth of use cases that people are doing with Chat GPT.

And tell us something about, you know, what does the world underestimate about Chat GPT that you have maybe been surprised by or listener might be surprised by?

There's been a real change in the way that people think of Chat GPT over the last year or so, where it's increasingly like a true thought partner to people.

It's not just a thing that answers your question, but it's a thing that you can, it's a sparring partner that you can actually think things through with.

And that shows up in all kinds of domains ranging from life advice, where you've got a relationship problem and you can actually get a lot of value to Chat GPT just helping you think through how to handle it and how to talk to your partner about it, all the way to a work setting where you're working on analysis or you're trying to figure out how to frame something or you're trying to build something.

And Chat GPT really shows up as a second brain of sorts.

And I think that's qualitatively different in terms of how the mental model it occupies with people.

And you see in the usage patterns in these cases that exist.

The more we nail things like proactivity, which we talked about earlier, tasks, etc., I think the more it's going to feel like a teammate in the workplace and like a super assistant at home.

And I think that's going to meaningfully change the use cases that people come for.

Yeah.

You know, I've been the most high stakes thing I do with Chat GPT is we have a new baby and the baby's crying at three in the morning.

Chat GPT.

What's going on?

First of all, congrats.

Second of all, I've heard this from all parents in my life.

The chat GPT has become indispensable as a thought partner.

And it makes sense, right?

If you have a really specific scenario or you think it's a specific scenario to you, Chat GPT really comes through and can help you build confidence.

And I think that's such an empowering thing.

Right.

And I imagine new parents aren't always the most confident about what is the right thing to do.

And if Chat GPT can make you feel like you are you have agency and control and, you know, I think that's really valuable.

Yeah, it's huge.

Well, thank you for making Chat GPT.

It's literally getting me an extra hour of sleep every day.

It took a village.

But that is a great metric.

That should be the North Star metric.

It's like incremental hours of sleep.

That's a great one.

And incremental hours of joy.

There you go.

I mean, you joke, but we talk about this a lot.

And because spiritually that is pretty close to what we hope we can do.

Right.

Is help you reach whatever you consider self-actualization, whether that's sleep or joy or any other goal you might have.

Yeah.

Yeah.

Well, thank you for thank you to the village.

We're going to switch gears and talk about the landscape.

Sure.

There's a lot going on on the field.

You know, how would you frame Chat GPT's differentiation to people out there?

There's a lot of different products out there.

Look, it's the best time in history to be a consumer of technology.

It is, indeed.

Because you've got options and the competition is intense.

And I think that's beautiful.

And it's actually good for us, too, because if you were to pre-mortem why a company like OpenAI does not achieve its mission, it's probably focused because of the sheer number of opportunities that become possible when you approach AGI.

And having competition and options out there, I think it forces us to focus on our customers, too.

And on things that really matter, which aren't always the most flashy thing.

Right.

So that's its latency, reliability, the quality of the user experience.

So I think it's a really good thing.

I think the biggest differentiation of Chat GPT is the team behind it, because we're not static.

Anything we build will get copied, sometimes in ways that are high craft, sometimes in ways that are sort of check boxes.

And it's really important to us that we evolve the category and build the super system that we've always imagined.

And I think the reason that I have confidence that that's possible at a speed that outpaces the dynamic of being copied is that we have an amazing team.

And that amazing team across research and engineering and design and all the different functions that it takes to make something amazing.

And I think our unique ability has been to bring those functions together to build something that is sort of at the intersection of useful and possible right in that moment.

So my best answer for you is we keep pushing forward and we hope to be expanding what people think of this product as.

You know, last winter we had obviously what was called Code Red.

Google had a great model.

There was a lot of talk about it, Mark Benioff switching very vocally to Gemini and us delaying ads and health agents and shopping.

Basically hit pause and everything, making chat to be better.

Talk to us about that moment, both about what led to that and what was happening in that moment.

Yeah.

So first off, Code Reds are a tool we use to create focus.

And as you can imagine, when you're in a place like OpenNed, this is what makes us special too.

Special to work here is there are so many different things going on.

It's a research lab.

We are pursuing many different ideas.

Right.

And there's been these moments where we've wanted the company to come together to solve a problem across boundaries, no matter what your project might have been.

And the last year we had one of those moments where we felt like we need to show up for users.

We need to focus the things, focus on the basics, like reliability, performance, the way that talking to the model feels, making personalization really great.

All these elements that our users care about.

And I loved it because it was really an opportunity to work with a bunch of folks who I don't normally get to work with on making the product great.

And we just exited the Code Red, which we knew we would, with the launch of 5.3, which is a great model for the everyday user.

It's great to talk to.

And 5.4, which is workhorse if you're trying to do real knowledge work.

And undoubtedly we're going to continue to use the tool of a Code Red whenever we want to create focus.

But I'm excited because I think Chat GPT is in a great spot.

Yeah, so Code Red is over now.

That's correct.

It's not the new normal.

It's not the new normal.

We want it to be a special thing, but it is a tool I suspect we will continue to use.

That's great.

That's great.

And maybe tangibly if you were to point out how did Code Red change Chat GPT or maybe the ops or how the team operates?

The thing I try to get, the foster with the team is focus.

So we're certainly more focused than we were six months ago on the things we really want to nail.

And some of those things are very behind the scenes, like latency, reliability, those kind of things.

And some of those things are very conservative efforts, like involving Chat GPT into the super assistant.

And so focus is the main lasting artifact.

And as you imagine, it's hard to stay focused sometimes when there's so much going on in the space.

But that's the hard job.

And you asked me about tradeoffs earlier.

Getting the team to focus on the things that really matter to users is certainly one of them.

That's always worth it.

In the back of my mind, as I asked you that question, is all the other founders that are in the arena right now.

And just a reminder that, hey, Code Red is a tool for you.

Wartime and Poundra, as you used to call it, is a tool.

Yeah, I think every company does it differently in terms of how you get stuff done.

But I think it's really valuable to have terminology that means something, that signals to people it's okay to drop your other stuff and it's okay to focus on this thing together, even if that wasn't your original job.

So I think it works really well at a place like OpenAI.

But I imagine startups would have an equivalent.

Yeah.

You know, one of the things that caught everybody's imagination on our team was what Peter was doing at OpenFlaw.

Incredibly potent to put all the tools together.

Obviously, Peter is a great builder.

Congrats on bringing on Peter to the team.

Tell us a little bit about what Peter is working on and when might the billions on chat at GPT have something to see there.

Well, first of all, I'm very excited for Peter to be here.

I was excited to have another German speaker in the house.

He's Austrian and German.

So we're exchanging Guten Morgens.

But the OpenClaw is so inspiring because it brought to life in many ways a vision that we'd had in different forms, admittedly, around this kind of AI that is fully embodied, that exists across different UIs that can do stuff for you that has an interaction pattern that feels a little bit more like talking to a human because OpenClaw allows you to interact in a very, very natural way.

We can send many texts back and forth and it's very current.

And so there's a lot of elements of OpenClaw that I think were very clarifying to folks across the industry.

But the best, you know, I'm super excited to just learn from Peter and bring into the company and figure out what we can do together.

So there's a lot more to come.

All right.

So now on to the most fun section, Rapid Fire.

All right.

You ready?

Sure.

We'll start with my favorite game, which is Long Short.

Pick an idea, a startup, a business, a product that you love.

You think you're very bullish on?

If I was starting a company today, I'm really excited about these companies that are going into companies and getting extremely hands-on and doing effectively professional services with the AI because we've saturated all the emails and you need to get proximate to the problems.

So it's those companies that I'm paying attention to.

Fascinating.

So this is an example to be like, hey, you're going and either acquiring or going inside an operating forum that has scale and a humming engine.

Exactly.

And making that a more efficient engine.

Yeah.

Or just like, you know, you're doing contracts for customers that have really hard problems and you're actually going in and committing to solving the problem.

And doing outcomes.

Yeah.

Because like, you know, there's a reason I think that we've made so much progress on math and coding, but not on many other domains because those are domains we are proximate to.

We as people who work in labs.

There's all kinds of other domains that we are not as proximate to.

And if you get proximate, I think you can build something transformative.

And I think this is more important now precisely because the easy problems have been solved.

The obvious problems have been solved by the models.

Credit where credit is due.

I think Notebook LM is awesome and differentiated and helps me learn new stuff.

I think it's great.

It's so good.

It's so good.

I think this is the example of you can innovate and you can build something totally different.

It's awesome.

Yeah.

Yeah.

Yeah.

It's so good, particularly for I found it for some more technical learning to be a very approachable way to totally learn.

And it's really cool.

I feel like AI and underrated capability of AI is to just transform things into a different medium.

And I think that's so important for learning.

We just launched these like dynamic math blocks, which allow you to visually understand math and such.

I should be to learning is always a big use case for us, too.

And I think just being able to transform things from text to visual soon from visual to video and like all these different media is amazing because people have such different ways of processing information.

And some people are like auditory learners and people are visual.

Some people like reading.

So I think that's really magical and a great, great angle to take.

Yeah.

Amazing.

Amazing.

Amazing.

You know, one of the things I think about a lot is education and education for kids now in school.

The world's changing so fast.

I'm not sure our education system is changing that fast.

What advice would you have for for students who are in school now who might have to adapt faster than the system around them might adapt?

It's a really good question and something that I thought a lot about myself.

You know, the I think the most important skill in this era is curiosity, I think, because if the machine can answer all your questions, you better have good questions.

And the only way to have good questions, I think, is to pursue the things you were actually excited about from an early age and throughout your entire life.

And I reflect on this because the only reason I'm here and working on this stuff is because I thought it was neat when I got, you know, nerds sniped in the interview process.

Right.

And it's like, this is so cool.

And so no matter what you're doing, I think that's an important skill is to be curious and learn to stay curious.

I think I'm confident that if you foster that skill, you will know how to adapt to, you know, an evolving landscape of tools and ais and jobs.

So that would be my advice.

Yeah.

Curiosity is has always been the premise skill.

Our friend Bill Gurley wrote about it in his book, Running Down a Dream.

But you have to check it out.

Yeah.

What is a job that gets more valuable, not less as I gets better as as I arrives?

Well, I think maybe the easy answer is being an entrepreneur because it's the best time to build ever.

And it's like being able to self actualize your your idea.

Maybe one that is maybe non obvious is I think writing actually is very important.

And it's not because the I can't write.

You know, I will become amazing at writing just like any other domains.

But because I think the skill of writing forces you to be very clear on what you have to say.

And even though prompt engineering is obviously going to go away and has gone away to much extent, the idea of expressing what you want to a machine requires you to be a pretty good writer and a very precise writer.

So I would say that that is it.

And any profession that involves very clear writing and therefore thinking, I think, is well set up.

Yeah.

100 percent.

Honestly, I mean, this is the whole thing about slop, right?

There's just so much.

That's the other thing.

I think there's going to be a permanent need for high quality, trusted, authoritative content and tools like Chachipiti can help you discover that content.

But I think the need for amazing content is also here to stay.

And final question, what has been your AGI and feel the AGI moment?

When did you feel it?

I've had so many, honestly, and it's.

It's definitely not stopped.

A few weeks or so after I joined open AGI, GBT4 had finished training and I remember trying it out and it actually it didn't impress me at all.

And nor anyone else that week because it kind of didn't work.

And it's because we hadn't figured out how to post train it.

And I think seeing it go from kind of wait, is this really a thing or was GBT3 kind of it to wow.

Actually, this is an entire step change with what felt to me at the time who didn't understand much about it at all as like some tweaks or some a little bit of final stretch work was profoundly humbling because you can realize that you might.

It might not look like we are close to really powerful, useful AI, but we probably are.

And then the moment that really there was two things that GBT4 did that felt like AGI to me.

One is it could do poetry and I didn't think it was possible for any of them to do poetry.

It just kind of fundamental philosophically.

It just didn't feel like in scope.

And then the other one was it could produce code that actually worked and compiled.

And then the next moment where I stared at the ceiling just in awe was when I realized GBT4 could just simulate an entire computer terminal like a full computer with commands, etc.

And I'm like, wait, how would this be imbued in a language model?

There's been so many moments since then, honestly, like reasoning was a moment.

One of the moments was when I think Mark and I were giving a demo of reasoning in front of the whole company.

And this was a moment where we were still trying to kind of find use cases that were hard enough for the for the reasoning to make a difference.

We're way past that point.

We know.

But at the time, I think we were having to do a puzzle in front of everyone.

I think one of the moments that maybe totally feel the AGI is like we were in the middle of the demo and everyone started laughing.

I was like, wait, what is funny?

And then I stared at the screen because we're showing this chain of thought as it was streaming out of the model.

And the model swore and said like, oh, damn, it may have to adjust because I realized I had made a mistake in the puzzle.

And the fact that it did that, but particularly the fact that it did that in a way that was entirely emergent from the, you know, our process completely blew my mind.

And, you know, made me feel quite humble about what else these models might be able to do.

So that was one of those moments.

And then most recently, watching people use codecs like watching people have walked around with their computer open because they don't want the task to end.

Watching people who have never coded in their life make stuff and bring ideas to life.

That feels like an AGI.

Honestly, it's just accelerating for me and it doesn't wear off at all.

And everyone has a different thing, obviously, but those are worth some of mine.

Yeah.

You know, it's 10 years ago, there was a product called Kite.

I don't know if you remember, it was for software engineers.

It was like an AI coding product.

That's when I felt the hunger for personal AI.

And nothing happened for 10 years and then everything happened for the last 10 months.

The timing thing is really hard because it's actually quite possible to predict where things will end up, I think, in terms of the kind of product form factors you're going to have.

But to know when it happens, it's really hard for me to make statements on anything between sort of eventually and in three months because of all the ambiguity around.

Well, that's a tight enough window.

Now in three months is a tight enough window.

Three months is pretty okay.

Try to stick to the three month plan more or less.

Though my team would probably tell me we don't, but I try.

But yeah, anything in between three months and eventually is difficult.

Yeah.

Well, thanks for doing it.

You've got a lot going on.

This was a total treat.

We're so excited to see all the great products you released for us.

If we can do anything to help, let us know.

Awesome.

Thanks very much.

Thanks for having me.

Of course, man.

This was fun.

As a reminder to everybody, just our opinions, not investment advice.