CoreWeave on AI-Native Infrastructure and the AI Engineering Loop

Source material

Full source text

[MUSIC PLAYING] Welcome to the Practical AI Podcast, where we break down the real-world applications of artificial intelligence and how it's shaping the way we live, work, and create.

Our goal is to help make AI technology practical, productive, and accessible to everyone.

Whether you're a developer, business leader, or just curious about the tech behind the buzz, you're in the right place.

Be sure to connect with us on LinkedIn, X, or Blue Sky to stay up to date with episode drops, behind the scenes content, and AI insights.

You can learn more at practicalai.fm.

Now, onto the show.

[MUSIC PLAYING] Welcome to another episode of Practical AI.

We are the podcast that tries to make AI practical, productive, and accessible for everyone.

We get to talk to all sorts of cool people in industry.

And today, I want to introduce you to Corey Sanders, who is the senior vice president of product at CoreWeave, which we've been seeing in the news quite a lot lately and looking forward to learning more about CoreWeave and the various problems it solves and the position in the AI world that it's fulfilling.

Welcome to the show, Corey.

Thanks, Chris, for having me.

I was going to say, you got to talk to all kinds of cool people, but today, you're stuck with me.

So here we go.

Oh, no, no.

That's funny.

I know, I got to say, you have a really interesting background.

I know with things changing as fast as they are, I know you did a long period at Microsoft before coming to CoreWeave.

And you have seen so much of the cloud landscape change.

I can't think of anyone better to address that in terms of going all the things you did in Azure and then coming to a little bit of a different problem set as I'm looking forward to hearing from you about.

And so anyway, if you could start us off and what problems are you interested in and paint a picture of how you see the world from where you're sitting?

Yeah, no, it's a great question.

And yeah, I mean, absolutely right.

I did 20 years at Microsoft before I joined CoreWeave and worked in the early days of Azure.

And so it's been a trip now to kind of take a step into CoreWeave and think about many of the same approaches we had with Azure in the early days now applying with an AI-specific lens.

And that's really sort of the right way to think about what I think CoreWeave is doing.

And sort of my view on sort of the world and the industry right now is we are seeing the opportunities for AI-based applications to evolve and grow into a similar way that cloud-based opportunities back in the early days grew and evolved, where it starts off with one or two things and suddenly people are expanding and growing and building.

And then seemingly overnight, obviously it took many years and I think will take many years this time.

Every single application, everything, single service, every single thing that we sort of touch and experience in tech has AI as a significant component of it.

And I think that pivot is a critical part of the places that I'm working, which is with that being a pivot of how applications and how services are built, the need for AI-specific capabilities, AI-specific, AI-centric application models are paramount.

Sort of asking people to apply that AI concept to their legacy way of building applications will end up slowing them down, end up creating challenges.

And so the opportunity and what I get to really focus on every single day is asking the questions, "Great, I understand this is how it was done, but how could it be done different with an AI-centric perspective or AI focus?

And how are we building these services to make it so that people who want to go build AI have a place they can do it where that's the entire focus of the platform?"

When you talk about that kind of AI-centric approach that you're taking with a lot of people, and we certainly are guilty of it on the show, like people here, for a long time, we were talking about the latest models coming out and we were in that and the last year, it's been a genetic everything and stuff like that.

And so can you talk, like there's a perspective shift here.

I think we're people when they're thinking AI and there's first things that pop into their head, can you talk a little bit about like what was missing?

As like, if you were coming from kind of the old way of doing cloud, what was missing that needed to be there instead of just as an add-on?

Because that's what happened early on is like, "Oh, hey, you can get a GPU in our cloud and everything."

But now with this kind of different approach that you've just talked about, which goes all the way from infrastructure to application, what has been missing that needs fulfilling?

How are you thinking about that problem?

And like, what are some of the details around what infrastructure should be going forward?

- Yeah, absolutely.

Well, maybe as a starting point, it would help to explain maybe the different sort of different streams of AI, right?

And you sort of mentioned agentic.

And so maybe I'll start there and talk 'cause I think it helps a little bit of a frame and the streams have separated and then there's a convergence now that we're heading towards, that's also worth talking about.

So look, I mean, at a very basic level, there are kind of two big aspects of sort of how AI is used or leveraged.

One is on the training side, right?

One is the actual creation of these models, right?

Creation of these weights, right?

The making, sort of doing the learning in a very similar way to the way the human brain works, but doing the learning to actually create the output that we then engage with, whether it be AI, whether it be anthropic, whether it be meta, sort of all of those have that sort of background in training and that requires very specific infrastructure to be able to deliver.

And the infrastructure is very expensive, right?

As we all know, and it's very deployed in sort of large tranches and very interconnected, right?

One of the biggest requirements for many of these training workloads is they're sort of these large tranches of deployments that are all deeply interconnected.

So they're all working together.

And as part of that training area, some of the things that CoreWeave identified and certainly even the broader market identified is the potential places where that work slows down.

Because the GPUs are so expensive, right?

Any sort of slowdown in being able to get the job done is impactful.

And so that includes things like failures in the GPUs.

That includes things like storage being loaded into the GPUs, right?

And so all the way down at the infrastructure level, what's going wrong with the GPUs?

Are there failures?

Into all the way up to the orchestration over which job is running on which infrastructure to be able to optimize the output, right?

And so this is where I think AI focused direction, that's a very different set of requirements than just your standard public cloud with a whole bunch of computer acts, right?

It requires a bunch of both design and how the hardware is being laid out and specifically how it's interconnected.

It's very unique to AI workloads, all the way up to then how the jobs are being orchestrated with knowledge over what's being run down at the core level of the platform.

And this is a bunch of places that I think, you know, CoreWeave has a bunch of differentiation on, things like our observability platform, our storage platform, right?

There's a bunch of places where we offer unique services targeting that type of AI massive scale training workload that then is just very different than your, again, your classic cloud workload.

So yeah, go ahead, pause there, 'cause the second half of the story I'll get to, no, but go ahead, Chris, I'm ready.

You had a follow-up and I'll talk 45 minutes uninterrupted if you let me, so let me pause and let you kind of jump in, go ahead.

- No, it's fine, and don't lose the second half of that story 'cause you had me, which is why I had the follow-up on there.

I am curious as you're talking about that transition, like if you'd bring, like for you personally, when you were, before you came to CoreWeave, when did you have that realization?

Like, is this, as whatever this epiphany is occurring, you know, was it all at once?

Did it happen over time?

Like, when did you realize I wanna do something different and don't lose that second half of the story that I was cutting off as I got to go?

- Yeah, no, no, no, right?

And the second half of the story is very similar to the first half, just a different sort of outcome, actually.

But yeah, I mean, for me, look, I got the honor of my final years at Microsoft getting to work on sort of a lot of the AI infrastructure aspects in the platform.

And it was kind of a funny thing because it was sort of not my day job, like my day job, I was working on industry solutions.

So like financial services, you know, retail services, et cetera, trying to think through how these large verticals could leverage AI in their workloads, which actually is the second part of the answer.

But the first part of the answer, I ended up getting pulled into a lot of the discussion over how, at Microsoft, will we optimizing the deployment of that infrastructure, right?

And that realization really struck me there, which was like, you know, prior to that, a lot of the big cloud strategy was you end up having a much space and power, and as the needs come up, you deploy what you need to, right?

So you need more compute in a given location, you deploy it.

And within, you know, days, you have a huge amount of additional compute, you need a bit more storage in a location, you can go deploy a bit more storage, and you can kind of add, and that sort of commoditization, that sort of fungibility is a big value add for the big clouds at the scale that they're running.

But it turns out you can't really do that when you're dealing with sort of AI workloads, again, all interconnected, all wired together with, let's say, InfiniBand, Iraqi, very specialized network behind it, very specialized storage underneath it.

Suddenly, it's like you've got to pre-plan things, right?

You've got to be thinking about it ahead of time.

And so it definitely was, it definitely for me was an experience of learning, right?

It started there.

And then as I moved over to CoreWeave, I think the realization of how many places from, again, caching, specific caching for AI workloads, to specific approaches to Kubernetes, which is, of course, a standard orchestration solution, but can deliver focused AI enablement for bare metal-based deployment to really squeeze out all the juice of that GPU, right?

All the way up to, again, the orchestrator being sort of aware, that realization of like how many aspects of the platform needed to be, could be, and then needed to be customized for AI-specific workloads.

That was really, I feel like I didn't fully get it until I came to CoreWeave, but I saw it and the beginnings of it when Microsoft, and I don't know whether that's a Microsoft or Core thing, or more just a point-in-time thing for me personally, like it's more probably that learning over time that you start seeing more and more of this customization, realizing, wow, like this is because of how much value and cost there is behind this, it is worth it to do this customization because people will pay for it to get the most out of those very expensive GPUs.

- So you raise a great question.

If you are, it's you or you, you're coming with the background that you have and the experiences that you had professionally, and you're making that adjustment, and you're seeing that in your company CoreWeave and see the capability, how do you, like as a problem, like you've identified a particular set of problems that are part of this evolution, and you're doing it from a great vantage point, given professionally where you've been.

So all the other AI practitioners that are out there don't have the benefit of your background.

So how do you teach them that there is something that they're not seeing that is a set of concerns that they need to address that they don't even know that they have?

How do you approach that education piece?

- Yeah, I mean, look, it's very hard.

By the way, it's very hard for me.

I mean, I think even so, as much as I appreciate your sort of lead in that I'm obviously uniquely capable at doing this, it's actually that experience, that sort of big cloud experience, while it's super, super valuable, it also ends up actually being a limiter because you come in with a bunch of assumptions, right?

To your point, you come in with a bunch of like, well, this is how we built it.

Like we learned that over 10 years and this is why we built it.

And you don't necessarily question all the things you should question over like, well, but maybe this is a different world, right?

And so I do think it's, in some ways it's innovator's dilemma sort of at the core, right, it is basically, and I say this a lot, like what got us successfully here today does not necessarily get us successfully to the next wave, right, and that to me is such an important, just life lesson, but it's certainly a tech lesson.

And so for me, like it really is like asking those questions, right, it's asking sort of the point of question of like, okay, so that's true, but that's true based on 10 years of commoditized, you know, general purpose public cloud assumptions.

What if we assume this is different here in AI?

What if we assume that the data requirements are different or the container's image sizes are different or the overall flow of AI innovation is different from typical software development.

Now what problems do we need to solve?

So in some ways it's all about asking the right questions and it's also one of the things I always like to push on as being, I think one of the most protected aspects of humans when it comes to the innovation of AI is that creativity, asking questions that maybe are different from the way things are.

As we all know, AI is all based on the way things are, right?

It is deep learning and regurgitation with a ton of amazing magic in what exists, right?

But sort of the ability to now say, well, but maybe what exists is wrong, that's the difference.

And I think that's where when we approach AI and frankly, when we approach a lot of tech innovation, that's the questions that need to be asked.

And then I would argue as much as I think I'm decent at it, hiring really good people who are amazing at it is actually my secret sauce.

So I look for those people in the market and basically try and find the people who are going to challenge me and say, well, but Corey, maybe your baseline is wrong.

Like this is what you did in Azure and it worked in Azure, but maybe it doesn't work here anymore.

And it's like, oh wow, that's good, harsh, but fair.

And so, you know, that type of approach I think is-- - Okay, so one of the things that you mentioned a few minutes ago, it kind of caught my attention and I wanted to go back to it and that's having to do with training.

And when you have training jobs fail and you talked a bunch about like where in the system that could happen, you kind of identified some of those points, but could you talk a little bit about like what actually goes wrong in the systems before an AI practitioner or researcher or something has identified that things have gone wrong and that they're not making progress along, you know, the line that they're trying to work toward.

You know, I think there's a little bit of, I don't know, you know, just black cloud mystique about like where that can happen.

Can you talk a little bit about what you've seen in terms of where that problem arises and how and what it is, what the implications are on that?

- Yeah, absolutely.

I mean, it's one of those special areas where there's both science and art to it, right?

I think, and that's, you find some of the best people in the industry who do this, it is that sort of instinct and sort of knowing how things work.

But like, look, I think that the concept of a failure in a job can mean a whole lot of things, right?

There's certainly sort of failure from an infrastructure perspective.

You know, the infrastructure fails, the sort of a node slows down, things are running slower than you'd expect.

And that can be sort of a plethora of things, right?

That can be a storage slowdowns, that can be GPU slowdowns, right?

One of the things that we've recently launched, which is a surprisingly enthusiastic capability is our GPU straggler detection, right?

This concept of like GPU slowdown for a variety of reasons inside their infrastructure.

And then when your job is hundreds or even thousands or sometimes tens of thousands of GPUs, finding that one slowed down and the output of that is maybe your job is slower.

But like, which GPU slowed down, right?

Like which GPU isn't performing?

And so that type of detection and observability is actually quite challenging.

And so, you know, a bunch of that observability, this is a big part of, I think, where that sort of AI-centric work, that sort of, you know, we believe our secret sauce, but certainly everybody's working on that sort of like AI-centric focus on sort of being able to detect those types of hardware failures, the hard failures, and the soft failures, the sort of slowdowns and so on, that can really dramatically impact job.

So that's kind of more on the like pure failure hardware impact.

There's also the other side, which is, you know, a big part of this, it is research, right?

I mean, and we call it research for a reason, which means you run a ton of experiments, right?

You've drawn a ton of experiments with a bunch of different parameters and different controls, right?

And then you need to compare them all and sort of study them and understand which ways went what and sort of now, based on your output, what should the next experiment be and so on.

And this is where, you know, a product like Weights and Biases, which is the top of our platform, really focuses on that.

But one of the biggest is sort of that experiment tracking and comparisons and so on.

One of the biggest things we've heard feedback on and recently sort of announced, hopefully a response to this is very new, is it's really hard.

You're point about sort of the art versus the site.

Like it's really hard that detecting and finding and then learning from that experiment what the right next step is.

And so we've recently launched what we call ARIA, Core Week ARIA, which is short, it's clever name, short for AI Research and Iteration Agent.

And its entire job is to try and do this sort of continuous analysis on experiments that are being run and what the right next step should be.

And so look, I think the simple answer is right now it is a lot of experience and expertise in this area that's driving that type of work.

But as is so common in technology, a big step of platforms like ours and others in the market is to enable that democratization.

How do we make it so that many more people can do that type of iterative model improvement work?

And using AI itself to do that, I think is sort of a clever approach.

- With you talking about ARIA, could you talk a little bit about like that, I presume is introducing a kind of new workflow.

New capability, it gives new insight, but there's new workflow there.

Could you talk a little bit about like how that changes the practitioners day-to-day activity and like what's introduced or taken away and kind of talk about the benefit of that.

- Yeah.

I mean, look, I think if you take my full sort of view, full extension of time, I think this concept of portal experiences and console experiences and sort of standard views, I think they're all going to fade away.

I think the experience is going to be an interaction with an agent, right?

Now how that interaction actually sits, is it in a website?

Is it on a console screen?

Is it in my VS code console?

Like I don't know, right?

Like I think there's a lot of different places or all of the above frankly.

But like I think the idea will be, hey, here is the type of improvement I want in my model based on what I'm seeing from my traces.

What's the best way for me to run some experiments?

Great, run them, right?

And then on my mobile app overnight, I see, okay, the experiments ran and the agent's popping in and saying, hey, the experiments ran, it turns out this work, this didn't, right?

I think we should do another run here.

What do you think?

Yes, let's go do it.

And so then that run kicks off, right?

And then it comes back and said, oh, this improved by quite a bit, but based on this data, perhaps it actually would be helpful to go and iterate and iterate and iterate and loop and loop and loop.

And so this sort of like approach to AI loop, I think will be that type of that development experience.

It will still be the brains of the researcher.

Like it will still be the, well, I don't know if that's gonna work.

Let's try this instead.

But it is going to be heavily guided by this sort of agent analysis of the data versus you go into your console, you look at the line charts of your six runs and you compare your success rate to each other and your performance rate to each other and you visually look at it, which is by the way, state of the art today, but not the way that the world's going to move.

And I'm excited about that direction.

I think that that agent led is the right direction.

And candidly, I would argue, I think that's going to be the case for all of our interactions moving forward, no matter what we're working on, whether it be banking or buying clothes, right?

It's going to be that type of interaction model versus click on a button that says buy, which I just, I think will be antiquated in five years and sometimes even laughable intent, but we'll see sort of how far it goes out quickly.

>>I know, well, I'm on the older side of things and can remember back.

And when I think about some of the things from a few years ago, you're like, wow, we got through it.

One of the things I wanted to ask about is you've mentioned agents a few times.

And I think typically people are thinking about agents in terms of, okay, I have my model and I'm going to figure out in my application, how many agents are assigned to the model?

Is it one-to-one or one-to-many?

And I'm going to give them the tests and give them the interactions.

But I'm wondering, is it a little bit different on the infrastructure side?

Like how do you see, you've kind of already made reference to it.

So like how does agent architecture work in an infrastructure environment like that?

What do they mean to you?

And what does it bring?

And can you talk a little bit about like, this is what we get on our backend as a provider versus this is what you as a customer would get on the front end.

Can you talk a little bit about that infrastructure, a Gentic story there?

- Absolutely, yeah.

Yeah, I mean, so I'll go back even to, I think the question I answered, I feel like many moons ago, where I said there were two paths, right?

And we sort of went down the first path.

The second path is this inference side of the house, right?

Which is basically the AI application, right?

I like to call it AI application because I do, to your point, I think you said like, everyone's talking about a Gentic and I think that's fine.

But like in some ways, it's all about an application that's AI centric, right?

And a Gentic happens to be one type of that, but all of it will sit on some aspect capabilities of an inference call, basically leveraging all those models that we just built and asking it questions or asking it to do an analysis or asking it to simply spell check something, right?

Like all of these things could be on the backend there.

And look, I think to your point, as we get more and more complex and there's in some ways two different approaches to AI applications, there's certainly the productivity internal facing ones, the co-pilots of the world.

And then there's what I like to call the sort of mission critical, the business critical applications that are either external facing for a customer, so like a customer's customers or sort of critical to their business workload, like a gene folding application for a pharmaceutical company.

Now that second category, and now kind of returning back to your most recent question, that second category, I think will be a very complex application model, right?

Like over time, right?

Today, I think there's a lot of this concept of like, I have a problem and I'm calling a frontier model and I'm getting an answer, right?

And that's my app.

But over time, it is going to be, I think, consist of many agents or many models, many inference calls of many different types that come together to build an application, right?

So in some cases, the very deep analytical, sort of the biggest hardest problems may go to those frontier models and call into them and ask them to kind of do that sort of analytical work.

But like translating, someone asks a question in Japanese and you wanna translate it to English, like that doesn't need that level of depth, that may enable sort of a different, smaller, cheaper model.

And I think as you look at some of the very specialized workloads, the gene folding example, it may need a very specialized approach to a model versus again, this general purpose feels like a big, a big giant hammer for maybe a screw, right?

And so, you can certainly push it in, but it doesn't quite work.

And so the opportunity, I think, for some of those workloads is to think much deeper about how their application is going to work.

And this is to your point, how is it going to leverage the infrastructure, right?

You know, what is it with the infrastructure being sort of a significant component of the cost, how is it leveraging it?

Is it fully optimized for that infrastructure?

Is it fully infrastructure aware?

All the things that I mentioned before about being infrastructure aware and storage caching and so on, all of that makes the inference workloads work better, work faster, work cheaper.

And then I think the other question is, is it leveraging the right model, the right parameters, the right prompt?

You know, and that to me is opening up a new approach to AI application development, which we like to call this AI loop, where you're never done making it better, right?

You're never done making the prompt better, or you may want to take a model and make it a little bit better and do a little bit more training on it, and it feeds back into that training component.

And so this concept of doing inferencing, running an AI application, having 30, 50 different models, all interacting with each other, and then finding the part that needs improvement or could be cost, could be a cost benefit, or could have a little bit better results, or could run a little bit faster.

And then running that through the system and using products like Weights and Biases to take that output, run it through the system, use something like ARIA to learn and make it better and better.

And so I think that's our future for those types of applications.

And that's all gonna be built on the best infrastructure all the way up to that workflow that I think is gonna be crucial to building the best apps.

- It seems like the amount of web pages, landing pages, et cetera, that we have to create every week just keeps growing.

And often those turn into a pile of tickets and handoffs amongst team members.

And that's why I love what Framer is doing.

One of our partners who has a pro website builder platform where teams can collaborate in real time and iterate on the same page and publish instantly.

And you can even tie in agents into this work where agents and humans work in tandem.

Agents bring speed and scale, people bring taste and judgment and control.

So I would encourage you if you are starting a new website from scratch or needing landing pages or relaunching a website, check Framer out, learn more how you can get more out of your site from a Framer specialist or get started building for free today at framer.com.

/practicalai for 30% off a Framer Pro annual plan.

That's framer.com/practicalai for 30% off.

framer.com/practicalai, rules and restrictions may apply.

- Okay, Corey, so as you were talking a moment ago in answer to that question, I've got a whole ton, a whole ton of questions.

- I'm sorry.

- You know, it's good.

(laughing) This is what we're here for.

But no, it's like, so as, you know, when people are out there and they're thinking about their own workflows and they're looking at, you know, the current incarnation of agentic engineering is kind of a loop engineering and getting all your loops going and everything like that.

And you made reference to that in your answer.

And so as people are trying to constantly level up in that way, and there's a learning curve for them to do that, what with CoreWeave, if they go, I'm moving, they've listened to you today.

They're like, I definitely wanna move to this platform.

What changes in their agentic or loop engineering that they should be thinking, if they're already trying on whatever platform, maybe Amazon, Azure, Google, whatever they're on, they're already kind of trying to do that.

As they come and get onto CoreWeave and you've talked through our conversation about all these advantages, all these different capabilities that you're bringing in, how does that change what they're setting up in their prompts and how they're setting up agents to address, how do they take advantage of that in their current workflow?

What adjustments are made?

- Yeah, I mean, I think that one of the key things is being able to quickly execute and learn from that loop without a lot of additional noise or disruption.

And what I mean by that is, and this is one of the things that I'm pretty excited about, what I get to kind of work on here at CoreWeave, is how are these, how is that loop all basically playing out in interconnected services?

And how do you basically execute against that loop?

And so by running your inference service and having it then leverage our Weave platform for traces and running evaluations, basically saying, okay, in production, I'm now seeing that with these types of questions or these types of operations, I'm either going down this wrong path, so I need to figure out how to fix that, or let's say it's taking too long, and so I need to think about, am I trying a different model?

And so the tracing from production allows you to sort of understand where you need to make these improvements.

And with the agent, sort of the ARIA agent, helping you sort of find those and say, oh, well, gosh, this looks like it's slower when it has this type of question, to then being able to directly execute, okay, great.

If I change my prompt, does that fix it?

If I change my model, hey, this new Keemie model is out, wow, like this I'm hearing is really, really fast, like, let me change my model and see if I get impact.

And then I can quickly set up an evaluation that then gets executed, right?

You can use our sandbox product to be able to just run that evaluation, say, okay, now I see these two together, and I understand now, okay, the performance did get improved on this one with Keemie.

Let me make sure that the pricing isn't sort of a big difference.

Okay, it's okay.

So now let me kind of swap that in and get it back into production.

And now let's take that loop again, right?

So the loops can be fairly small, right?

Like every single modification, in fact, most modifications I'd say are not, let me put this into my training workflow and actually do some reinforcement learning or do some fine tuning.

That is a possibility.

Like in some of these cases, like, okay, gosh, this is great, it's working well, but like my accuracy is just a little bit off in these types of questions.

And I don't think a new model is gonna help.

Let me actually do some fine tuning, right?

Or let me do some reinforcement learning since I maybe don't have the data to do my fine tuning.

And you can run those services again, all integrated into Cori with again, the weave traces to see, did it work, run your evals, it did, plug it back in, right?

And so like I think what we're going to find is that your developers, at least what I believe developers are gonna look for is one, being able to do much, if not all of it, all in one place, because just the simplicity of being able to track those traces, having a registry, which, you know, Cori Boffers for lineage, like every updated model, every updated parameter, you know, et cetera, being able to travel, that version was three yesterday, okay, that's the one I want.

Within an agent that sits on top with the knowledge of like, oh, that's what you need to go try or you should go work on that.

To me, that's I think the real power here.

And I'm excited that like, you know, these components, like Cori delivers a bunch of this, but a lot of this is also, we're continuing to, you know, to make it super easy for that flow to happen at a much more accelerated pace than it even happens today on Cori Candles.

>> Gotcha.

With, you know, as you're talking about like that flexibility to get different models, but in the same conversation we've been talking about kind of just all the benefits and the virtues of the performance wins you get from that kind of vertical integration of your stack.

Is there any kind of tension between having, about kind of supporting an open approach on one side and having it deeply optimized in terms of portability, you know, for a customer and things like that.

How do you, you know, what's your position on the tension there in terms of portability versus performance optimization?

>> It's an awesome question.

And, you know, I'll give an answer that I've actually believed for quite some time.

I had the luxury of deploying the first Linux infrastructure on Microsoft, it was called Windows Azure at the time.

So it was Windows Azure hosting Linux.

And so, you know, my belief system from then to today, and I think the CoreWeave belief system is we need to expect and build for multi-cloud.

Like this belief that, you know, CoreWeave is going to be the only cloud people use for their applications, I think is false.

Like I think, you know, and so I talk about this loop and I'm enthusiastic about the loop, you know, having all the components on CoreWeave to deliver on this loop.

But with the strong realization that people are not, that many people are not going to use every service that CoreWeave offers for the loop, they're going to use some other services.

They're either already have, you know, an evaluation service that they really love, right?

Or they already do their training somewhere else and they want to run sort of the inference here.

Or they already do their inference somewhere else and they want to run their training here.

I also think in a, so I think being very open and delivering upon consistent services is a key part of our strategy, right?

And that's sort of like when you think about things we offer, Sunk, which is Slurm on Kubernetes.

Slurm and Kubernetes are both, you know, open source projects, right?

And so, you know, we take a lot of pride in believing we deliver the best.

And that really to me is where I think the differentiation and value comes in is can we take some of these open source, multi-cloud, cross-cloud capabilities and make people want to use us because we offer the best services.

And so a lock-in with love is sort of the joke I typically make.

And so, you know, I think that really is how I think about it.

And so it's also is why we take some of our Sunk anywhere we recently announced.

You can actually take our Sunk offering and deploy it on other infrastructure.

Like we think it works best on our infrastructure.

Yeah, go ahead.

>> Go ahead and go there.

No, it's just because I go there.

You've mentioned Sunk a couple of times along with Slurm and Kubernetes.

And like go there and tell me a little bit about it.

And add in also, you know, for people who are using, you know, Kubernetes in their old world, the old cloud and stuff, like what's different?

What's it doing well?

What does Sunk bring?

Can you kind of package that up so people can kind of follow you there?

>> Yeah, absolutely.

Yeah, so, you know, put simply like, I think Kubernetes is such a market leader and there's deep awareness and understanding on how to operate and orchestrate infrastructure, right?

And workloads on infrastructure, right?

The concepts around Kubernetes, sort of the platform that's been built on top of it, certainly has effectively dominated that market for years, which is awesome.

And then Slurm, which is kind of a job scheduler, orchestrator placement engine that has, you know, I would argue been a dominating force in research-based workloads, right?

In sort of AI-based research.

It's not the only one, right?

There's quite a few out there and we could have probably a full hour show debating the merits of each, but many, many AI researchers know Slurm and are comfortable with Slurm.

The challenge is that the Slurm on top of infrastructure is actually tricky to manage, right?

Slurm on top of infrastructure, particularly for sort of AI researchers, is challenging sort of managing ups and downs of infrastructure, failures in infrastructure.

And these are the things that Kubernetes is awesome at, right?

And so what Core we built was Sunk, which is short for Slurm on Kubernetes, although it's with a U, and it is about bringing the best of both worlds, right?

Enabling the sort of the power, the capabilities of Slurm job scheduling with the orchestration power and ease that Kubernetes brings and sort of jamming them together and creating sort of a platform.

And this is sort of a key product that we offer directly on our infrastructure.

But like I said, we have now recently announced that we offer that to be taken anywhere.

It's called Sunk Anywhere, it's clever.

And so you can deploy this on other clouds too and take the power of that Slurm plus Kubernetes support onto any cloud.

>> So really, really interesting.

And like one of the questions I really wanted to get in for, I'm taking a slight curve on this, is as you are servicing and providing more and more capability to people, people are always talking about the cost of things.

I don't mean the cost of any given provider, but just in general, to be able to do meaningful experimentation and research at the frontier is so, so capital intensive in terms of the compute and stuff.

And like, how is that something that could be addressed?

How do you guys approach it?

And how do you, and in a broader context than just CoreWeave, how do you think the industry at large needs to approach the problem?

Because it kind of, there's a special few in a sense that have a lot of resources and a lot of others are saying, how do I compete against that?

>> How do I do that?

>> A lot of those folks are listening right now and watching the show and like, what do you, what would you say to them?

How would they approach that and what can CoreWeave do?

>> Yeah, I mean, I think the part of the goal with this AI loop concept, part of the goal with this sort of integrated story is to enable people to do more with less, right?

Or do the same with less, I guess, depending on how you look at it.

So, and so what I mean by that is, optimization can mean a lot of things.

So it can mean being able to choose a cheaper model that gets the work done well enough, right?

And so it requires less infrastructure, requires sort of a less deployment model.

It may be doing model improvements and fine tuning to reduce the size of a model or enable it to be more effective with a smaller sort of scale.

Part of the goal with ARIA and things like ARIA are to enable less trial and error, right?

Enabling faster movement from experimentation to production, which is actually a big component of that cost construct, which is like, great, like I want to go build some production, but how expensive is it going to be for me to get there?

And with something like ARIA, with something like the AI loop, you have sort of this flexibility to get things out, get them to start be revenue generating, sort of get them to start building on your business and then experiment and learn as you go.

And with the help of like, hey, I wouldn't spend the money to fine tune this.

I think you could just shift your prompt and improve it, says agent.

And so like, I think the opportunity to sort of get that guidance to help direct, that is built on the back of what, you know, the learnings we've gotten with all of those massive labs still working.

By the way, those massive labs are still on the very cutting edge frontier where like an agent that talks about historical knowledge may not be as helpful.

But for everyone who's kind of one step behind the frontier labs, it's by the way, I think every enterprise and every sort of large company would want to be there because they're like here now, right?

That is where I think some of these capabilities coupled with sort of the metal based optimizations that I've already talked about, where I do think this type of platform could really be meaningful.

>> That's really cool.

With another thing that is, you know, so big this year and exploding outward being kind of embodied agentic engineering at the edge, you know, robotics is just taking off, you know, with AI integrated.

What is CoreWeave's story going forward in that space as you are getting, you know, where you're not strictly based on cloud, but there's some hybrid between cloud and edge platforms that are out there with different levels of embodied intelligence and agentic capabilities.

>> Yeah, I mean, there's a few places that we really focus when it comes to, you know, robotics and, you know, let's say some of the scenarios around robotics, industrials and manufacturing and so on.

You know, one is, I talked a little bit about, you know, our models, our experiment tracker technology, right, and I kind of talked a little bit about looking at the different charts and so on and so forth.

You know, one of the things that we've recently added to our weights and bias platform is a specific robotics focus, right?

And something even kind of as trivial, but is a huge, huge value is shifting from those sort of line charts showing success to visuals showing success, right?

Showing the actual robot and how it's moving with all of your experiments and being able to detect what went wrong and what didn't.

Obviously, this is pre-agent where I do think then add on top of that ARIA and you sort of have all that benefit of that robotics sort of training and iterative work adding up to being able to sort of drive a bunch of the robotics workloads.

And then, you know, the other area that I think is quite interesting is work with Monolith, which was a company we acquired a few months back.

And, you know, one of the things that we've realized in many of these cases is the need to go engage directly with customers on the types of scenarios they're building, right, and where we can go help.

And so that's where, you know, I think a lot of this sort of focused engineering resources going in and sitting with customers and saying, "Let's talk through what you're building.

"Let's talk through what problems you're trying to solve."

And let's figure out how we can help.

And, you know, we've had this for a while as part of our overall mission control offering.

We called it Direct to Expert, which is really where we kind of lean in with resources and expertise.

That sort of additional Monolith brought some of the industrials, the manufacturing expertise to bear here.

And I think it's a big factor in sort of helping people build up their solutions and we'll continue to look at other verticals as well.

And then the last point, you know, as we mentioned, I think more and more of the capabilities that we're building, we are enabling to run on other infrastructure, right?

And so I mentioned Sunk already, our caching service for our storage platform can run on other infrastructure.

And so most of these are based on just Kubernetes operators, which means an edge environment, an on-prem environment.

Like these also have the opportunity to leverage these same services and take advantage of them in those types of environments.

And so, you know, even our services, our software solutions, our ways to advise the solution, all can be brought to bear in those environments as we see some of that evolving and maturing.

- That is super cool.

As we start to wind up, long time viewers and listeners will know that we ask you to start waxing poetic a little bit in the last question.

And that is, we always like to get your, what we describe is like, the day's over, you've chilled out, however you chill out at the end of the day, you know, you kind of get in bed, your brain spinning on different problems and what might be.

And if you can share a little bit of your vision, your dream about where you think things are going, it might be for CoreWeave, it might be for the broader industry or both, but just like what's in your head for like, for, you know, several years out towards not on your product map.

You know, I'm not asking for what's your next release.

I'm asking for like, where are things going in the large in your view?

And recognizing that just as I will do this and get it wrong regularly, it's fine to get it wrong, but I love to hear people thinking at any given moment about where things might go, whether you get it right or wrong.

So what's yours?

That I'm chilling out in bed before I go to sleep?

- Yeah, I mean, I probably say, and I talked a little bit about it already early on the call, but like maybe double down on it.

Like I think the progression that we saw with public cloud to the point now where every enterprise, every company in the world has a public cloud engineering team.

Like it's like, in fact, they far exceeded their on-prem engineering team in most cases, right?

There's obviously exceptions to that, but for the most part, like everybody, and you're sort of, you know, every single application or experience out there has some backing in the cloud.

I think we will go through the same progression with AI and probably faster, to the point where we will have AI engineering teams at every single enterprise company, and they will, probably sooner rather than later, exceed the amount of people who are on the sort of cloud engineering team, right?

Cloud engineering teams will be legacy and the AI engineering team, and they will be the supporting team for the AI engineering team who's getting to build the AI applications.

And AI applications, I think, will far exceed our current interaction model with AI applications, right?

Like I think, and I'll use sort of, again, the cloud is my example, like the initial wave of cloud was click a button, click a button, click a button, right?

But now we record podcasts on cloud-based experiences, right?

We watch, you know, World Cup games on cloud-based infrastructure.

Like I prefer that over going to my TV and being like, "Gosh, what channel is this on?"

Right?

And so that transition, like, it will be much more than just I chat to an agent, right?

It will be a visual response to the face you're making to your screen when something comes up.

It will be, you know, human interaction models that will far exceed what we experienced today and will replace what we know of as modern today.

And I think it's going to happen much faster.

Cloud took, what, maybe 20 years, 15 years to kind of really get there?

I think we'll probably half it, right?

I think within, you know, five to seven years, that's going to be our world.

And websites with, you know, buttons that you click are going to be legacy.

Like I can't believe they haven't AI-ed this yet, right?

Or whatever the verb may be.

And so that to me is the direction and the requirement there for a company like CoreWe, but frankly for a company, all the companies in this space is how do we enable that progression by democratizing what's required to get there?

Because right now, those enterprises don't have those people.

They don't have those frontier lab researchers, right?

They don't have the talent that they need to be able to be innovating in this way.

And so they're limited by it.

And so just like they didn't have the cloud people in the original world, right?

And so they will ramp up, they will train, they will skill, they will grow, they will hire, right?

That's going to happen.

But in parallel, just again, like the cloud way, we have an obligation to democratize the services so that more people can deliver upon that value for less.

And that to me, like I think of why I'm here and I don't even mean here CoreWe, but I mean in the industry right now, that is why.

- That is very, I really appreciate that insight.

That was not only do I definitely think that you're onto something there, but it's interesting to listen to and fun.

Thank you so much for coming on the show.

That was a great conversation.

I learned a lot.

So Corey Sanders of CoreWe, hope to have you back again.

- Appreciate it.

Thanks for the time, Kristen.

Come back whenever you want.

(upbeat music) - All right, that's our show for this week.

If you haven't checked out our website, head to practicalai.fm and be sure to connect with us on LinkedIn, X or Blue Sky.

You'll see us posting insights related to the latest AI developments.

And we would love for you to join the conversation.

Thanks to our partner Prediction Guard for providing operational support for the show.

Check them out at predictionguard.com.

Also, thanks to Breakmaster Cylinder for the beats and to you for listening.

That's all for now, but you'll hear from us again next week.

(upbeat music)