CoreWeave on AI Loop and Agentic Infrastructure

Source material

Full source text

I know I'm on the older side of things and can remember back.

And, you know, when I think about some of the things from a few years ago, you're like, wow, we got through it.

One of the things I wanted to ask about is you've mentioned agents a few times.

And I think, you know, typically people are thinking about agents.

I'm thinking about agents in terms of, okay, I have my model and I'm going to, you know, figure out in my application, you know, how many agents are assigned to the model.

Is it one-to-one or one-to-many?

And I'm going to give them the tasks and give them the interactions.

But I'm wondering, is it a little bit different on the infrastructure side?

Like, how do you see — you've kind of already made reference to it.

So like, how does agent architecture work in an infrastructure environment like that?

What does it mean to you?

And what does it bring?

And can you talk a little bit about like, this is what we get on our back end as a provider versus this is what you as a customer would get on the front end?

Can you talk a little bit about that infrastructure agentic?

Absolutely.

Yeah.

Yeah.

I mean, so I'll go back even to, I think the question I answered, I feel like many moons ago, where I said there were two paths, right?

And we sort of went down the first path.

The second path is this inference side of the house, right?

Which is basically the AI application, right?

You know, I like to call it AI application because I do, to your point, I think you said like everything's, everyone's talking about agentic.

And I think that's fine.

But like in some ways, it's all about an application that's AI centric, right?

And agentic happens to be one type of that.

But all of it will sit on some aspect capabilities of an inference call, basically leveraging all those models that we just built, and asking it questions or asking it to do an analysis or asking it to simply spellcheck something, right?

Like all of these things could be on the back end there.

And look, I think to your point, as we get more and more complex, and there's in some ways, two different approaches to AI applications, there's certainly the productivity internal facing ones, the co-pilots of the world.

And then there's the what I like to call the sort of mission critical, the business critical applications that are either, you know, external facing for a customer.

So like a customer's customers, or sort of critical to their business workload, like a gene folding application for a pharmaceutical company.

Now that second category, and now kind of returning back to your most recent question, that second category, I think will be a very complex application model, right?

Like over time, right?

You know, today, I think there's a lot of this concept of like, I have a problem, and I'm calling a frontier model, and I'm getting an answer, right?

And that's my app.

But over time, it is going to be, I think, it consists of many agents, or many models, many inference calls, of many different types that come together to build an application, right?

So in some cases, the very deep analytical, sort of the biggest, hardest problems, may go to those frontier models and call into them and ask them to kind of do that sort of analytical work.

But like, translating, someone asks a question in Japanese, and you want to translate it to, you know, English, like, that doesn't need that level of depth that may not, that may enable sort of a different, smaller, cheaper model.

And, you know, I think, as you look at some of the very specialized workloads, the gene folding example, it may need a very specialized approach to a model versus, again, this general purpose feels like a big, a big giant hammer for maybe a screw, right?

And so, you know, you can certainly push it in, but it doesn't quite work.

And so the opportunity, I think, for some of those workloads is to think much deeper about how their application is going to work.

And this is to your point.

What, how is it going to leverage the infrastructure, right?

You know, what, what is the, with the infrastructure being sort of a significant component of the cost, how is it leveraging it?

Is it fully optimized for that infrastructure?

Is it fully infrastructure aware?

All the things that I mentioned before about being infrastructure aware and storage caching and so on, all of that makes the inference workloads work better, work faster, work cheaper.

And then I think the other question is, is it leveraging the right model, the right parameters, the right prompt?

You know, and that to me is, is opening up a new approach to AI application development, which we like to call this AI loop, where you're never done making it better, right?

You're never done making the prompt better, or you may want to take a model and make it a little bit better and do a little bit more training on it.

And it feeds back into that training component.

And so this concept of doing inferencing, running an AI application, having 30, 50 different models, all interacting with each other, and then finding the part that needs improvement or could be cost, could be a cost benefit, or could have a little bit better results or could run a little bit faster.

And then running that through the system and using products like weights and biases to take that output, run it through the system, use something like ARIA to learn and make it better and better.

And so I think that's our future for those types of applications.

And that's all going to be built on the best infrastructure all the way up to that workflow that I think is going to be crucial to building the best apps.