OpenAI Launches Codex: An Autonomous Programming Agent

Source material

Full source text

huge news!

OpenAI is launching a new coding agent.

It’s called Codex.

Codex is a web-based software engineering agent that’s designed to work pretty much autonomously, so you can have it working on many different features and bugs, in parallel, while you watch.

I’ve been using Codex for a couple days, and it’s pretty great.

So I invited Alexander Embiricos, a member of the product staff at OpenAI who’s responsible for Codex, to come on the show and talk to us about it.

We’re going to demo Codex together and talk through all of the product decisions that lead to its release.

We walk through Codex screen by screen, and he’ll talk to us about how OpenAI is thinking about agents, and what OpenAI’s vision is for the future of programming.

Let’s jump in.

Alexander, welcome to the show!

Hey, thanks for having me.

So for people who don’t know, you are a member of the product team at OpenAI, and you are one of the people responsible for building Codex, which is a new programming agent that is launching today.

Well, we’re starting, technically we’re starting yesterday, but it’s launching today, Friday.

I’m super excited, but I got to try it.

I have a review coming out along with this podcast, and I just wanted to go through it with you and understand all the things that you put into it and all the ways that you’re thinking about it.

So thank you for coming on the show.

Yeah, excited to be here, and thank you for being an early tester.

Awesome.

So what I want to do is just give people right off the bat a sense for actually what it looks like, and we can sort of talk about it together.

So this is Codex.

Turnate process is great.

This is Codex, and it’s a coding agent, right?

So I can type in a task, like, "Please replace the headline on the homepage with 'Codex is out now.'"

And I can just press code.

And one of the really cool things is it doesn’t go into an agent experience.

It just adds it as a task.

So it’s basically built to make me just, you know, create a bunch of tasks and then not look at them, more or less.

That’s how I feel about it.

So tell me about, like, this screen and how you thought about this and why you did it like this.

Yeah, so I’m so excited that you’re screen sharing this and showing people so we get to know about all the details.

But, you know, for anyone listening, basically Codex is a cloud-based software engineer that can work on, like, many tasks in parallel.

And there’s a ton of AI tools out there.

I feel like programming with AI is completely different now than it was, like, two years ago.

And this form factor that we’re looking at is kind of the beginning of us thinking about what is it going to look like even, like, a year from now, today.

And the big shift is, you know, today when people are accelerated working with AI, a lot of it is, like, very, like, a very tight feedback loop.

It’s, like, very collaborative, like, a lot of, like, really good tab completion, right, and, like, really good chat.

And sort of where we see things going is we have these really awesome reasoning models.

And we want to give them time to think.

And we want to let them use tools safely so they can do more stuff.

For example, like, running commands to execute tests to be more sure of their changes and then iterate.

And so we basically want to give the agent its own computer to do work in.

And so as we started thinking about that, you know, you kind of realize, okay, this feels a bit like delegation.

And I'm sure we'll get into this.

But we tried a bunch of different form factors for what it would feel like to delegate to an agent, including, like, maybe the most obvious one, which is just, like, talking to chat GPT and having it do stuff.

But the short version of it is what we realized is that engineers wanted, like, this really functional tool to be, like, very efficient in how they delegate.

And so the thing you see here, it's, like, you know, it's very structured.

You've got the list of tasks that you have.

You can see how many lines were changed in each.

You can see what the status of the PR is, which is ultimately the thing you're going for.

And so this ended up just being a thing that we iterated our way to that we thought was the best first version to ship.

But actually, you know, I would love it if we end up talking a bit about what it might look like in chat GPT as well, because we definitely plan to bring it there.

I would love to do that.

And I have a bunch of follow ups on that specific point about, like, sort of the chat GPT versus what this is point.

But so to give people, like, a sense for how this actually looks.

So I did a couple things yesterday.

I actually, like, shipped a feature on one of our products that I've never coded in.

Like, I've never talked about repo.

And I shipped a feature to prod yesterday with Kieran is the Kieran is the GM of that product.

And he's a really, really talented engineer who's also early an early tester of codecs, like he was watching me the whole time to make sure I can mess anything up.

But it was really cool.

And basically, like to show people what it looks like.

We have this we have this UI in this is not the right one.

You're looking for the diff, right?

Yeah, where is it?

You should be able to click on there.

Yeah, nice.

By the way, this UI that Dan's showing us is being like editing, like changing massively, like every single day as we race towards launch.

So you know, thanks again for testing early.

Yeah, this this may not be what you actually see.

Or depending on when you're listening to this, it definitely is not going to be what you see.

But for now, for people who are listening, basically, I clicked into one of the tasks.

And you can see, basically, for this feature I wanted to build, we have this we have a view that I wanted to create a persisted collapsed state, we wanted to be able to like collapse a particular view.

And I just said, I just said do that, basically.

And you can kind of see a log of all the things it thought while it was doing the task.

And then it gives me like a summary of what it did, I can see it diff, and I can also just push that right to GitHub.

And one of the things that I noticed is the summary is very concise.

And in talking to Kieran about the code that it generates, the code seems very minimal, or like terse.

Can you tell me a little bit about how you thought about making this thing code and talk about it's talk about what it makes?

Yeah, it's a great question.

So, Codex, the agent runs on a custom model that we trained just for this product, called Codex One.

Codex One is a version of 03 that's optimized for real world software engineering, but not only optimized for real world software engineering, but also optimized for this form factor, which is, hey, it's gonna go off and do a bunch of work, and then you're gonna get a diff back.

And once you're using this a lot, actually, you're gonna be getting many diffs today back.

And I'm sure this thing we talk about a lot with even human software engineers, like, would you rather review like 10 100 line diffs, or 20 50 line diffs, or 1000 line PR?

Right?

Definitely the thing is you don't want to review the 1000 line PR, right?

And so, as we trained this model, we both wanted to make sure it was great at coding.

And so the model like slightly outperforms 03, actually on like evals like Swedish.

But more importantly, actually, we wanted it to produce like code that was really mergeable, and then talk about the work that it did in a way that was like actually really reviewable for the human reviewer.

So, we put a lot of effort into style here, you know, making sure it's not doing things like adding extraneous comments, making sure that the style that it's using is actually like the style in your code base, and not like its own style that it has and thinks is correct.

Okay, and so getting to your point about the PR descriptions, you know, early versions we've had of tools like this or models that did this would tell you everything about what they did.

And, you know, just actually to give you a bit of the story, like in our UI, we would like show you kind of the model's thoughts first, and then we would show the diff underneath.

And whenever like a full time software engineer was trying to use this a lot, they're always like, man, I just want to see the diff, like, I don't want to read the description until I see the diff.

And if the diff is good, then maybe I'll look at the description.

And so we kind of ended up realizing like, hey, there's actually like kind of a spec for what a good PR description looks like.

And that's like pretty concise, and only maybe explaining like at a high level what it is, the non obvious parts, like you definitely don't want to duplicate content that's in the diff.

Then the other thing we realized is that because it didn't run on your computer, you didn't necessarily see each iterative step.

And so you really want to know how validated this has changed, and how did the model try to validate the change.

And so you can see here on the left side, the model is also describing the testing that it ran.

And here, we put a lot of effort into first of all, just making sure that it's like never hallucinating, but also making sure that it tells you when tests passed and when tests failed, which is like super important in a concise way.

And then lastly, perhaps the thing I'm most excited about, like if you might mouse over that little like terminal icon next to the testing.

Yeah, so they sort of like the last thing on the third line there.

So the little icon is like a carrot.

Yeah.

So we'll actually, the model learns how to cite its output.

And so it's not just saying like, "Hey, here's a test I ran."

In the case of tests that passed, which it didn't here, it's not saying like, "Hey, the test passed.

Believe me."

It's saying, "Hey, the test passed.

Here's like a deterministic citation that we did to actually pull the part from the log out so you can review it yourself and be super sure."

And in this case, the test it tried to run failed, and it's kind of telling you this because maybe you would want to run it yourself or upgrade the environment it's operating in so that those tests can be run.

And it's giving you the output there so you can go and review it and improve it.

That's really interesting.

This particular repo, so actually, the one we're looking at, I kicked off the collapsed state task in the wrong repo at first.

So this is the wrong repo.

This is an internal app we built, and so it does not have any tests.

And I was just kind of curious what it would do.

If I gave it a task, it was totally not possible to accomplish and it bound a section, something else, to collapse.

So that's kind of just like an interesting little detail.

But yeah, okay, that makes a lot of sense.

One of the things that I notice about this that feels different from a Devon, for example, is I don't see this.

It has an environment that it uses that it sets everything up and it runs a test and all that kind of stuff, but it doesn't have a browser, for example.

So it's not going and logging in to the app to check if it's working or not.

Tell me about that decision.

Why'd you do it that way?

Yeah, I would say that's more of a just sequencing decision rather than a vision decision.

Ultimately, the thing that we view ourselves as building super long-term is just one super assistant.

And it has a bunch of tools and you don't even have to decide if you even want to use code mode or if you want to use newsletter publisher mode.

You just have one assistant, chat chat chat and you can just ask it for stuff and it'll use the appropriate tools.

And it'll also just like answer quickly if that's the right thing or it'll do work if that's the right thing.

So long-term, that's where we're trying to get to just like one thing with all the tools.

But what we're starting with is kind of like, hey, we want to ship a research preview.

We want to iteratively deploy this out in small steps as capabilities grow and we're going to start the smallest, most constrained thing possible.

So for instance, it doesn't take multimodal inputs yet.

We'd obviously love to add that.

It doesn't have a browser that would obviously be useful for validating front-end changes.

It doesn't even have network access.

So the way that it runs is so when you kick off a task, we set up a container, pull the repo into that container and then with network access, we run some set of commands that like you wrote.

So it's not the agent running those commands, it's like you're running those commands.

And so that can pull in like any dependencies like NPMI, NPM install or whatever you need.

And then actually we cut off internet access.

And I think we updated the logs recently to show that.

So if you were to click into logs and scroll to the top, you would see no internet access all the way at the top.

And then you would see no internet access.

Yeah.

So we ran the environment setup that you put in there and then we turned off internet and then the agent runs.

And that's just for absolute maximum safety.

I haven't actually ever seen it happen, but there's a theoretical risk or a tail risk that there could be some exfiltration.

Maybe the agent wants to go to Stack Overflow and ask a question about some code and then paste it in.

That's a theoretical scenario.

And so we're starting really small in scope and then over time we'll start adding more and more capabilities.

Got it.

And that actually brings me to my next thing, which is one of my gripes about this.

The fact that it's not in chatgbt means that it feels like a very one shot type user experience which to me makes it feel very...

It's definitely for senior engineers where you can see the entire thing you need to build in your head and you just type it out and then it's done.

Whereas for me, for example, I might want to go back and forth and chat with it to figure out, "Okay, what is it that I'm actually trying to build?"

Or in this case, in this particular session, I asked it to build a feature and then I realized after I built it that I needed to add something else.

So I asked it a follow up.

And in the follow up case, it didn't work quite as well as the one off one shot thing.

So tell me about that.

Yeah, it's really interesting.

Delegating to agents requires a bunch of mindset updates that actually I think will take time for people to learn how to get to the most use out of them.

And we'll end up temporarily in a state where we have tools that feel more like you're delegating and tools that feel more like you're collaborating.

And then our goal at least is bring those together so you don't think about it.

Again, I think about our future super assistant.

I don't think about it until I'm delegating.

I just ask it for stuff and it just does stuff.

But people are pretty used to working collaboratively with tools locally on their computer right now.

And in fact, we have the codec CLI, which is exactly that.

So you can do it back and forth with CLI, have an excuse to some commands.

It'll ask you for permission because it's on your computer, but you can sit with it and pair a program with the CLI.

So a little bit about the mindset shift that we've seen people who get a lot of value out of codecs like the agent that you delegate to.

I'll just share for because it's kind of interesting here.

One of them is like an abundance mindset.

So something that we've seen a lot of people who are using this a ton do is they just fire off many, many, many tasks without thinking actually too hard about whether it's going to work or whether it's perfectly described or not.

And whenever you need to do something new, you just kind of fire off a new task.

And so for example, a really common use case is on call triage.

It's like maybe you don't know what the bug is.

You were talking about how like, "Oh, I've noticed it works best if I can kind of see through the full arc of the feature."

You don't need to know that.

You just fire it.

And then sometimes it's like, "Hey, here's the exact fix."

And you're just like, "Wow, amazing."

Not only did I save myself time, but that bug was fixed faster than I could have ever gotten to it otherwise.

Other times maybe it just provides a draft and then you have to land it in your own time.

And other times it just maybe finds where the bug is or helps you just reason about it.

And so it's kind of like this idea of like, "This is a service to me that accelerates me to developer and I'm going to use it without necessarily always knowing if it's going to provide a mergeable PR or not."

That's on call triage as an example.

Another very analogous place we see people use it is like every morning people will kind of think about what they want to do and then they're at home.

They'll just be on their phone, fire off some tasks.

We have mobile support coming soon.

But we already support mobile web and mobile app on iOS.

We'll support soon.

So they'll just be like, "This is what I want to do.

Fire off some stuff."

Then get to their desk and look at what they want to use.

So it is quite one shotty in the way people use those two examples I gave you.

So I think you're right about that.

And so we've started poking at this like, "Does it need to be one shotty?"

And so you saw in the UI, when you send a task, you can choose if it's going to be in ask mode or code mode.

And code mode is definitely the thing we've spent the most time on.

But we're starting to experiment with making ask mode faster.

And so one way that this could evolve potentially is when you want to do what I just described, like fire off a one shot or a few shot thing, you basically ask code mode.

But if you want to be a little bit more exploratory, you can try ask mode.

And so I don't know if you tried this, but one thing you could do is ask for some recommendations using ask mode.

And that should be relatively fast.

It'll probably finish pretty quickly during this call.

And then we could try kicking off some of those tasks if you want.

Okay.

So what recommendations should I ask for?

Yeah.

I don't know.

Is there a class that you find very complicated that you would love to refactor in some code somewhere?

I wish that I was familiar enough with this code base to tell you.

Okay.

What about just suggest...

Let's run two prompts.

And again, this is abundance mindset.

Let's run one like suggest places to add documentation.

That's a safe one.

Actually, a lot of people use it for documentation and it's really good at documentation.

And then let's do another one which is like find some bugs and suggest some tasks to improve the bugs or to fix the bugs.

Okay.

Cool.

I think that this is really interesting.

I want to pause here for a sec.

The sort of abundance mindset that you're talking about where...

Yeah, you can just direct intelligence at a problem and you don't have to have it that well formed.

And if it doesn't work, it's fine.

And that's just a different way of thinking about how to program or how to manage a resource that I think is really interesting.

And yeah, if there's some sort of priority bug that's going on, why not send off three different workers to figure out while you try yourself?

Because if they fix it, that's great.

And I think obviously there's always a time and a place for understanding the full system and maybe stepping through it step by step.

But it's not every time.

And that's just an interesting shift.

It's a different skill it seems like.

Yeah.

I figure you probably use ChachPT Deep Research on occasion.

And if I think about how some of my habits have changed, I deep research a lot of things that I wouldn't have spent...

I don't know how long it would take me to do a deep research, but probably quite a long time, like hours.

And now I read sort of deep researchers on custom queries that I have way more often.

It kind of lets me be more curious in a way.

Yeah, many things a day basically.

And I think there's kind of a similar theme here with agents in general.

Yeah.

It's like you find you have more questions to ask when it's really easy to ask a question and get a really good answer.

I find that too.

I'm just...

I've learned way more because of O3 and deep research and just AI in general.

What about...

I feel like there's a tension here that I sense in this product and I've sensed myself in trying to build these things.

There's a sort of tension between making something good for a particular situation.

For example, this is great for picking off a bug or a feature.

But you kind of get this loss of flexibility where it's not, for example, it's not nearly as good at just doing follow-up questions and chat GPT is just great at that.

And so how do you think about that when you're designing something like this?

This sort of trade-off between specializing and the brittleness or loss of flexibility that you get?

And how do you manage that?

Yeah.

I think the long-term goal...

I'm sure you hear this a lot, but it's the G and AGI.

It's to build something pretty general and a lot of specialization, at least when it comes to building UI, is a bunch of design decisions that we have to make.

But if you're just talking to an entity, it can make a bunch of decisions about the right behavior and just be general for you.

And so where I see things like this going and maybe just working with agents in general is you can have a conversation with an agent and that should be an incredibly general thing.

Like I was mentioning earlier, it shouldn't only be for coding.

It should be an agent and it should be able to work across any modality, whether you're in your car or at your desk.

So that thing stays general and it should be able to do all the things it needs using tools, et cetera.

But when we're professionals doing a specific thing, like right now we're recording this podcast and this is some pretty specialized UI, but it's really good.

It would be quite annoying to have to use a chat interface to mute myself or adjust my volume here.

And so the way I think about it is there can be nearly an infinite number of bespoke, possibly even AI generated UIs that serve the specific purpose of the user who knows exactly what they want.

And so for me, the most interesting thing about this Codex Research Preview isn't actually the UI, although don't get me wrong, we've spent a little bit of rewriting it massively as we speak.

But the most interesting thing is actually training a model that's designed to work more independently and make the most of that asynchronicity and its own compute environment.

And then building the actual compute environments that we can scale up and let people set up the agent to have the tools that it needs.

So once you have those two things, the model and the compute environments, we can then start bringing that UI to everywhere.

So like I mentioned, I would love for that to be more readily available, just like if you're talking to chat chat chat, I would love for that to be available also closer to where developers spend their time.

So if you spend a lot of time in terminal, then the Codex CLI would be a perfect place for you to be able to delegate to that agent.

Similarly, if you spend time in your editor or in your CI or in your issue tracker, all of these places should be places where the agent is just sort of ubiquitously available for you to delegate to it.

So let me just push on that a little bit.

So in a perfect future, basically I'm talking to chat chat chat chat chat chat chat chat chat chat chat chat chat chat chat chat chat.

And then handling the follow ups be like, actually, I want you to do a little bit better.

And I'm maybe not even talking directly to a chat chat chat chat chat chat chat chat chat chat.

Yeah, that that I think that is one possible future.

And I think that is a future that you might use if you're like on the go, like on your phone, not at your desk, or, you know, say, I don't spend like a ton of time, like on marketing, but I want to like interact with some like marketing tools, like maybe I go through chat chat because I don't have mastery of the the actual like underlying systems.

But if I'm like a full time person who spends a lot of time like coding, or you know, whatever thing that it is that I'm doing, I probably have a lot of like mastery that I've learned over time.

And that also brings me great joy in my life, hopefully, to spend time with these tools.

And so then I would like go straight to the tool.

And the I should be like just available in that tool.

Yeah.

So I totally get that especially for the for like the interface layer and like having specialized interfaces for specialized tasks make total sense.

What about the model layer?

Like, obviously, the goal is to have a perfectly general model that no matter what you ask it, it can like just fulfill the request.

But also, it seems like, for example, when you do, you know, you did fine reinforcement learning on this model to make a better more sort of like senior software engineer type personality.

But even within that, like, this model clearly has a different personality than cloud code, or it's like the underlying model for cloud code.

And so like, how do you think about that?

There's, oh, there's always even if you're trying to be as general as possible, there's always these sorts of like trade offs between making it good for that thing and making it generally, you know, it's it's it's flexibility in other areas or its personality.

Absolutely.

Yeah, I mean, I feel like at OpenAI, at least we've kind of done this like a few times now where we've shipped chat LGBT, a ton of people use it.

And then we get some feedback from a specific audience we care about.

So like with GBT 4.1 developers, and we just, you know, spend a bunch of time with them.

This is a different team.

But, you know, they spend a bunch of time like talking with developers, understanding their feedback, and kind of like creating like a different set of evals that we realized we like wanted to improve on we cared about.

And so then we had this decision, right?

Like, do we try to, you know, get a bunch of like improvements into the mainline model, which takes time and has a bunch of trade offs?

Or do we kind of want to speed run it on the side, ship a separate model, and then like reintegrate some of those changes.

And I think our belief as a company is that like, there are going to be times where we kind of want to speed run something.

So like, often just maybe even for learning purposes, just to like have the flexibility like really go deep with a set of customers and like to kind of like generate the evals that they have in their heads.

And then from there, like ship it validate and then take those configs or the you know, those changes that we're making to the model and remain line them and when it like it seems like whenever we can do that, the whole overall system gets better and maybe even more powerful than a custom side thing that we've done.

That's really good.

And like what that makes me think about because one of the one of my other like little gripes with this is it reminds me kind of operator where this cool thing but it's like separate from chat GPT and like it's I don't think that they've you guys have released an operator update since the first one.

And so it just feels like it's like hanging out there and not necessarily getting improved or not.

I'm not using as much as I am using like sort of base chat GPT.

And so is that kind of the like one version of the vision is obviously like codex stays stays as a separate thing.

But then you integrate all the learnings back into like the base 03 that I use inside of chat GPT.

And that's how everything codes this together.

How do you kind of make sure that you're not building like this balkanized thing that's kind of forgotten about in you know, three months or whatever?

Totally.

Yeah, man.

I have so many so many fun thoughts here.

One thing I'll say is that yeah, we have a lot of sort of like these capabilities that we've talked about like being kind of like different and separate like are coming together and like that's the thing we think a lot about and it's I'm very jazzed about it.

So excited to share more as that happens.

But let's take let's take codex the in my mind here, there's there's kind of like two interesting things.

The first is like what we said, which is like, hey, anything we learn as we like build this codex model is like really helpful to us just as we think about agents in general and like, you know, like this idea of like, say like citations, right, or like describing your work really efficiently for the user to read after you've done a lot of work, like tuning the steer ability so that it like kind of makes the right amount of assumptions, which I'm not saying we nailed perfectly.

But like, that's a that's a theme like that's a generalizable thing for agents and like something that's going to be useful in general.

Similarly, and I don't know if you have this in your briefing, but we are planning to release an update, an updated small model for use in the codex CLI.

And, you know, there's a lot of I mean, it's pretty obvious, right?

There's a lot of overlap between these two models.

And then actually, you know, a lot of the learnings that we have from like both of those codex models, like we plan to use even in like mainline models, right?

So there's a lot of that generalizing that comes back.

I feel like your question also, though, is a bit about user experience and like having to go to operator or having to go to codex.

And so like, you know, I think codex is kind of interesting because it's maybe one of the first products or features we've built into chat GPT, where we have a professional audience that like spends a lot of time in their tools every day.

You know, like a lot of how like chat GPT overall is quite a general tool.

And I think this is like maybe some of the first time that we're getting like fairly specialized.

And so my view here is that like, we're not exactly sure what this should look like.

I think like chat GPT a year from now where we have more of this will look fairly different from chat GPT today.

But the goals we'll have in mind will kind of be these things that I've been saying, which is like, I think we're trying to build a generalized assistant and we cannot lose sight of that.

But we should give ourselves the room to give users the like power UI that they want.

So for instance, just to give you an example, going back to your opening question about the UI.

It's a list, right?

And it's a list and it has features like archive.

Like why do we have that?

Well, it turns out that people internally who use codecs use it a ton every day.

And when they're using it, and it's just like mixed in with chat, it's a little hard to find your things.

And then, you know, there's kind of this workflow you develop where you basically you're like, hey, I have an idea, like, for example, like, so I'm on the product team, I mostly do small changes for people make fun of me affectionately at work.

But you know, this string should be different or there's this bug here, right?

And if the bugs really hard to fix, I'm like, that's vegetables, I'm not going to eat that.

Oh, you know, put it in a small niche to you.

Yeah.

But if it's like an easy bug, you know, I'll wham it, I'll explain what the word wham is later.

But that's our code name for the project.

And so, you know, I'm whamming a bunch of stuff.

And then I don't want to track it separately.

Like, I just want to have this list of things.

And then like, when they're done, my workflow is basically look at each one and make a decision on if it's mergeable, or if I want to like pull it open to my computer, fix it, maybe change it.

Or if I'm just like, this is over, like, I don't have time to do this.

And so that workflow really needs that UI of like, what did you open a PR?

What is the status of that PR and the ability to archive so that I'm only looking at the things that I my current tasks basically.

So that's just an example of like the power user UI that makes a lot of sense.

Yeah, we feel important.

Yeah.

Yeah, I feel like chat is only appropriate when it's incredibly under specified what you actually want to be doing.

Whereas as soon as you know exactly what you should be doing, like chat is not the right really the right interface.

I'll add one thing to that, which is actually the other cool thing about chat is that it unlocks a lot of like emergent behaviors that are just mind blowing, right.

So when I like, I'm obviously really excited to bring to bring us back into chat as well.

But you know, one experience that I had when we prototyped in chat was I had a I had asked for a change on front end.

And actually, when I was getting started on this project, I had been I had spent I spent like five years including before open AI just like writing native macOS apps.

And so you know, I'm not super familiar with the latest, you know, tailwind, whatever stuff.

So I asked for this change.

And I there are a bunch of like classes were added in line.

I don't even what is this class?

So I asked, but we were in chat.

So I just replied with what is this?

Can you explain it to me?

And because we were in chat, and the context of like what was changed was just like in line in chat, chat replied quickly, which was the thing you were saying is one of your gripes that this doesn't reply as quickly, right?

Chat replied quickly.

And it asked me, do you want me to generate an image for you?

And I was just mind blown.

And I said, absolutely.

And then it generated an image.

And you know, I was very happy.

And so I think I think it's important that we always like make sure that like the overall assistant has access to these tools.

So we can experience things like this, and then like work those into the more functional like you guys as well.

How do you feel about and with your own personal experience and what you see internally or I don't know other beta customers or alpha customers that are using this like how this changes what it is to be a developer what the developer experience is and I'll give you an example.

In my testing over the last couple days with Kieran, again, he's a GM for Cora.

He's you know, super technical.

I'm like technical but like in a I can just build stuff to make it work kind of way.

And what was really interesting is we just sat together on a video call.

We're both screen sharing.

We both had a bunch of agents up and we were just like talking about how we wanted to make the product better, Cora better.

And then he would just say, okay, I want you to like, you know, build this little thing and I'd be off being like I want to fix this little bug and both of us have like these different agents all working on different features and bugs all at the same time.

And we're just chatting while we do this like we have the brain space to be able to like talk about what's going on.

You know, an agent would come back then we'd like kind of look at it together and be like, is this good?

Is this not good and then either edit it or merge it and it was just like a very different it was like social coding in this like weird way where we're just like chatting together and work is getting done, which I've never really experienced before and feels like a new model for building things that was not really possible previously.

And I'm kind of curious if that resonates with your experience or if there are other things that you're noticing about what it how this changes what it is to be a developer what developing looks like.

Yeah, that sounds pretty fun.

I didn't even say even like before we had any, you know, like before Codex I've already started feeling like coding is like more social.

It's like even if it's just me coding with an LLM.

And yeah, you know, like that startup I was till I mentioned that I was working on like I built the first prototype for one of the things that became it in the airplane without Wi Fi.

And I would never do that today.

Like if there was no Wi Fi, it's just over like I'm going to watch a movie, I just can't do it.

And you know, I do think this is we're kind of on the precipice of never like change of similar magnitude.

So the vision for like what it should feel like to be a developer, if we get things right, and maybe what it feels like in general to like just be a knowledge worker, the vision of my mind is you should be able to do the work that you want to do.

And maybe that's because it's worked, it's difficult to automate, which is a lot of work.

Maybe it's work that's like very ambiguous, that requires a lot of like complex decisioning, or maybe work that you just want to do for fun, or you want to be creative.

And like that's the stuff you do.

And I think it's really important that we invest in making sure that when the when you're doing the work yourself, you're maximally accelerated, right?

So like things like the Codex CLI that you can pair with, I think are critical for us to invest in.

But then the lion's share of kind of like known work should just be done by agents that you're delegating to.

And so I think this kind of like shifts where we spend our time.

And maybe we're spending more time in planning, or like thinking about what to do more time in design, and then more time in like validation as well.

Now, I do think it's interesting, like, you know, right now, I think, at least for the next couple months, we are going to also be spending time thinking about how to set up the environment for agents.

And so it's not only like, I think we're only going to be spending our time like being creative, I think there's also going to be a lot of like, it's a bit like becoming a manager, and you're thinking about I have this team, and how do I enable my team to be productive?

I mean, and it creates it like creates new problems that you still need to solve.

So like a really interesting example that I'm noticing is you can implement so many features that the features start to crowd in on each other.

And the product is just feels like less well thought through because like you have so much throughput, you just go like do this little thing, and it just takes 10 minutes.

And then you've done that like 50 times.

And you're like, this product is like way too bloated.

And no one's like, step back and then like, how does all this stuff fit together?

And that I think is like, we need new product management or hygiene practices for we have so much capacity to build stuff or fix bugs.

Like, how do we deal with the overproduction of features almost, you know, which is a new problem.

Well, I have some taste on this.

So I don't think this applies to everyone.

But at least I'm happy to hear what you're saying in that I think like taste is like fun to exercise.

Yeah.

It's like, and also like great fun for humans, you know, so it's like, I'm excited, like, oh, we can produce too many features.

And now we have to like choose which ones I'm like, this is a great future like, wow, product person's dream.

But obviously, I know that doesn't apply to everyone.

But for me, that that sounds awesome.

The other thing that I like hope that happens is that we actually just build many more software products.

And I don't mean like many more features, but like many more apps.

And this again, is this is just like my personal opinion.

But I really like these like small, like beautifully crafted apps that have like a small number of features.

And, you know, and sort of almost the more niche the use case, like kind of the more fun it is to use them for a little bit.

So like, you know, an example in my mind that I've like poked at, but haven't really spent time on is like maybe like a texting app, but for just me and my wife only, you know, like that would just be kind of a slightly ridiculous thing to consider doing while you have a full time job like a while ago.

But like, I would love to have that app.

And like, it doesn't have to have many features, but it's just for me.

Right.

And I think that like, as software development gets like more and more accelerated, like my hope is that because it's like, we can produce more of it, we end up actually using much more, but in much more bespoke ways for people.

That makes sense.

Yeah.

And I think you can see that in a history of other art forms, like, you know, photography or illustration, where 150 years ago, like you had to do like woodcut blocks, and then we figured out how to like do mass reproduction of the photos.

And that's just, there's a whole, yeah, it's a similar kind of thing where now we all have a camera in our pocket or we can do studio like generations or whatever, you know.

Well, yeah, now we're putting a podcast together, right?

Like for on a like specific topic.

Yeah.

Exactly.

So I want to go back to our ask query came back a while ago.

So I'm gonna make a note about that.

So we asked it to suggest documentation.

And so it gave me some interesting things that said the repos guidelines, I'm sorry, documenting Ruby classes with yard and stimulus controllers, the apprehensive JS Doc blocks, seed development guide, and then just found in some places where yard documentation is lacking.

Yeah, so this is real quick.

This is close.

This is not what I was hoping you would see.

We clearly have some prompt tuning to do between now and tomorrow.

Because let's can we check out the other one real quick?

Because so this is good.

But what what I would love to see in there is that it actually stubs out tasks for you.

Oh, there we go.

This one did it.

And I think that's because we said suggest some tasks, like we put those words in the prompt.

And so it's kind of similar, right?

It's answering kind of the question, but it's giving you these buttons, you can click to like actually do the thing if you want to do it.

Interesting.

Okay, so I'm seeing so we said find some bugs and suggest some tasks to fix the bugs.

And it says issue number one, and it gives me some relevant code.

And then it has a play button on the code, which if I click it, just does the changes that it's suggesting.

Exactly.

That's right.

And so this is us beginning to like, sort of play with this, this, this gripe that you have, right of like the sort of the interaction loop.

And, you know, the us to be anything like, yeah, how can we make it so that it's like, if you want to be more quickly like sort of collaborating with a model to figure out what to do, you can, you can do it.

And also, in a way, how can we encourage it?

How can we get this going back to like a lot of what we're spending our time thinking about is like this future where agents are writing a lot of code, like, how do we make that work?

Well, you probably would rather like get a large refactor done in a bunch of small PRs that don't have merge conflicts and all independently compile.

So for instance, like a common use cases, if there's a large file, you like suggest several tasks to break this down into a smaller class, and then you can like do the refactor and like a few small buttons.

So basically, this is this is something we're thinking about.

And like, I think long term, like our goal is to like, converge a delegation and pairing real time pairing into just like a single experience.

That's interesting.

Okay.

And, and we only have a couple minutes left.

So I really need to ask like, how do you see the agent landscape evolving?

So there's a lot of agents feels like every programmer is working on their own agent.

There's like the big labs like cloud codes got, you know, clouds got their agent.

There's, there's like a ton of startups, there's like the dead ends of the world.

And there's a couple that are coming out soon, like we're testing like four or five of them internally.

And so how do you think about this, like codecs versus every other agent, the difference in the positioning or the functionality?

And then do more generally, like how do you see that market evolving?

Is there like, is this sort of like a one agent to rule them all type situation?

Or do you think there's going to be sort of like this ecosystem of agents with different personalities?

Yeah, I mean, I just think this is an incredibly exciting time to be doing software development.

Because there's just so much innovation happening, and there's like so many good products out there.

So personally, just like really excited about that.

I think the things that were, were really focused on and uniquely positioned to do well, or like, well, the fact that we basically can train the model, right?

And we can go as deep as we want into making a model that's like, as good as possible for the specific use case, right.

And so like, the things that I'm most excited about us doing are like things like, well, you know, obviously having like really good coding intelligence.

But then beyond that, like really thinking about like, okay, but you know, this is going to be used by an engineer delegating to many agents in parallel, like, how do we make that mergeable?

Like, how do we get the style right?

The instruction following the steering, you know, have its site, its work, stuff like that.

And so I think like for us, like really leaning into the model is like definitely like one thing that we're going to continue to do.

And then I think the other thing is like, just like thinking about like, this scalable compute infrastructure, it just happens to be something that we're like quite good at here because of the way we do training.

And then lastly, like for me, and this might be like a little bit more out there, but like ultimately like, so the reason that I work here, you know, during the benefits of AGI to all humanity, and I think of the shape of that, that's not like some like ultra bespoke thing.

It's like this general, like AGI super assistant.

And so like, that's kind of the world that I want to live in.

Like, I want to wake up and just like, well, probably not immediately, maybe go for a walk first, don't look at my phone, but then at some point, I want to just like reach out to an assistant, like have a conversation with it and like just have one thing, and it does stuff.

And then I can like dive in and do stuff myself.

And so I think like, that's, I feel like the chat GPT is like as an app and as an organization is like very like AGI super assistant pilled.

And so like, for me, that's kind of where we end up bringing this towards.

Yeah, that makes sense.

The integration on chat GPT is the big thing that's going to make this very different from any other, any other tool like this.

Yeah, that GPT.

Alexander, it was so great to chat.

Thank you so much for coming on.

Thanks for building this.

Thanks for letting me try it early.

I'm psyched to see where it goes.

Cool.

Yeah.

Thanks so much for trying it early.

Thank you for all the feedback.

Hopefully you noticed many, many of you, the issues are addressed.

And yeah, thanks again.