NVIDIA's Bartley Richardson on Why 'Agentic AI Is Next-Level Automation'

Source material

Transcript

[Music] Hello and welcome to the NVIDIA AI Podcast.

I'm your host, Noah Kravitz.

Agentic AI is on the tips of everyone's tongues right now, it seems.

But what is it?

What makes agentic AI so exciting, and what should AI leaders like CIOs and IT execs be thinking about when designing an agentic AI system for an enterprise?

Here to break it down for us, live from GTC 2025, is Bartley Richardson.

Bartley is senior director of engineering and AI infrastructure here at NVIDIA, where he leads agentic AI and cybersecurity AI.

Previously, Bartley was a technical lead on multiple DARPA projects.

He holds a PhD in computer science and engineering with a focus on AI.

And he's here right now.

Bartley, thanks for taking time out of a busy GTC week to join the podcast.

Thanks, Noah, I really appreciate it.

That's a great intro, by the way.

I'm going to bring you along with me everywhere I go, and you can do that intro every time.

I'm anything I can for the cause, happy to do it.

So maybe we can start with to put you on the spot, you don't have to define things, but talk about agentic AI, what it is, why is it so exciting, you know, in this context of enterprise leaders.

Yeah, you know, I feel like what we're really good at in the tech industry, and I'll include us at NVIDIA and that and everybody, right, is we're really good at making things very complicated.

It's one of our primary concentrations, right.

And when I talk with people about agents and agentic AI, what I really want to say is automation.

Like that's the word I really want to use.

But automation, you just get half of it out of your mouth and people fall asleep, right.

So we call it agentic and agentic AI.

And really what it is, it is that next level of automation.

If you think about the evolution of automation, and how we were doing things manually, and even if you go back to factories, right, and that these types of things, these, I don't want to say mundane, but I obviously just said it, but these kind of like everyday repeatable tasks, right, we've come up with better ways to do it, right.

Like, example, I grew up on a farm in the middle of Ohio, and we would dig post holes by ourselves, right, these terrible tools, we have technology that does this now for us, right.

Agents are very similar, they're just instead of working in the dirt in the middle of Ohio, they're working on massive petabyte scale data silos, or, you know, information repositories to take that data, turn on a little bit, do this like mundane task, and then instead of returning data that the human has to go back through and look at, actually take that give it context, put synthesize it together with other types of data, and make it a little bit more actionable, a little bit more easy for the human to digest.

Right.

Can we talk about reasoning?

Yeah.

Is there a genteck reasoning?

Is it a thing now?

It's a thing, right?

Like, I think we're going to make it a thing, right?

You know, it's definitely a thing.

Reasoning is another one of these terms that gets, you know, kind of bended about quite a bit, right.

And really what you have is this kind of different type of model, right, almost, where you have a, I'll say a traditional LLM or an old school LLM, I don't know what that means, like two years ago, five years ago.

Like, yeah, these old school LLMs, and they're really good at kind of like this token prediction.

They can do the thing where, oh, okay, I, you know, I have the sentence, complete the sentence, and all of these types of things that are going to image task.

And that reasoning models are kind of like this next level.

And they've been trained and tuned in a very specific way to think, almost like think out loud if you look at their text and look at their structure, where they're going through and you've given them the tasks.

They're like, oh, I could do this.

I could do this.

I could do this.

I could do this.

And it's kind of like when you're brainstorming with like maybe your colleagues or even your family, right?

You're like, oh, we could do this.

We could do this other thing.

Reasoning models have that type of feel to them, right?

And so putting them in a place of a system that's like, Hey, I need to, I don't really know what I want to do, but like, I kind of want to make a plan.

Here's my, here's my loose guidelines.

Reasoning models can very easily explore that space for you and get to like, here's some options.

So maybe we can dive into the technology a little bit and you can kind of go through some of the components involved.

There's different video technologies here and just talk a little bit about each one, what they are, what they do rather, why they're important.

Yeah.

And I think the way you set that up is really great because it is this collection of technologies, right?

When we talk about agentic system, it's not just like, oh, I just completely a hundred percent brand new thing that we're trying to get this agent name.

Right.

Exactly.

Right.

Exactly.

It's, and I, you know, you tell people it takes years to be an overnight success.

Right.

And so it takes years to do what we're doing in agents and agentic systems right now.

And it starts with something like a data ingest, right?

And data ingest and being able to retrieve that information.

So if you look at things like NEMO retriever, right?

That's a great place to start.

We have a capability that now NEMO retriever doesn't just ingest text, right?

Like unstructured text, right?

Right.

Yeah.

But we can ingest multimodal documents.

So everyone's favorite documents type, the PDF, right?

Everyone loves it.

Right.

Great, great document.

Easy to work.

If you would have told me, uh, 10 years ago that I'd still be working in PDFs, right?

Like, I don't know.

Right.

Uh, but, but anyway, here we are.

And they can have all of these different modalities in it.

It can have pictures.

It can have structured images, which are like tables and graphs and charts.

They can have unstructured images and all of these types of things and extracting that information and keeping the context.

I'll give you a really simple example where let's say you have a graph, a bar graph, a simple bar graph.

And if you were to just give that to a really kind of basic computer vision parsing model, it would give you back and be like, yep, I understand that I have five rectangles.

Okay.

It's like, no, you don't because the right, just having five rectangles doesn't make that interesting.

Right.

So it's the relationship with the five rectangles to them.

It's the caption that goes underneath it.

Is there text somewhere else in this document?

It's not even on that page that explains that chart.

So you have all of these different, we talk about the, the, um, retriever and ingestion process.

We have all these different models, these NIMS that go into the pipeline and then we stitch them together to make this type of parsing possible.

So we have models that detect bounding boxes of images.

We have models that will like work specifically on texts.

We have models that look at captioning and put all this together.

There's like three, four NIMS, right?

That just go into this one process.

And what I love about retriever is, and you're going to see the theme, right?

It's built for enterprise quality.

And so if you look at PDFs, you know, if you're the average person, you have like a couple, you're doing research, something like that.

You have 10 PDFs, you upload them, you know, Carol, it takes five minutes, you know, whatever, come back and do it.

If you're an enterprise, right?

Like, you know, how many PDFs we generated in video?

Like, I think we could run the world on these PDFs that companies generate and you need speed, you need consistency and you need accuracy, right?

And so if you look at something like retriever, we're, you know, 15 times the amount of throughput that you can get in other like leading systems, right?

Fantastic.

With a really complicated PDF, we can go, you know, 10 pages per second, right?

Like through on like a single GPU instance, right?

This extract this, um, it's really cool that, you know, 50% fewer errors in accuracy compared to other systems.

So it all starts, if I think about agents, it all starts with that part, getting the data from human source into an embedded kind of source.

Right.

Right.

Right.

Uh, should we talk about agent ops tools?

Yeah.

Agent ops tools also.

So I like the progression that you're going right here.

So I've got my data, right?

Like, uh, you know, I've got it in there.

We're pretty good at that.

Agent ops tools are another way of saying this is kind of like a flywheel.

Yeah.

Right.

So you've got your, your system and we'll talk a little more about the system in a little bit.

Uh, and you might have your models and what agent ops tools are really good at is fine tuning, honing, and making that system even more efficient.

Okay.

Right.

Uh, so, you know, for example, you can through successive iterations of fine tuning through a new motion ops tools, you can have a, you know, like a 10 X reduction in your model size, right?

By fine tuning that down, distilling it right.

Right.

A little bit, which obviously translates to either speed or translates to energy or translates to money, right?

These are kind of currencies you can exchange, right?

Depending on what you care about almost 4% increase accuracy by doing this fine tuning model.

And the real cool thing about agent ops tools is it just kind of sets there, right?

Like, and you hook it in, you know, you've got your inputs, you've got your outputs, you hook it into your outputs, it feeds back into your inputs and it'll prompt the human.

Like every once in a while, it's like, Hey, what do you think about this?

And you know, those, uh, you go to any tool and you give it the thumbs up, you give it the thumbs down, right?

There's that, but what's even better is you can give it a free form texts.

Like not just did I like this or did I not like this?

It's like, Oh, you get feedback.

Yeah.

This wasn't quite like this, whatever.

And it keeps that context and we'll use that to like steer in a different way in a different direction.

Right.

And so it's useful in a lot of ways, model reduction size, all that, but then you've ingested all this data, you're using this model.

It's a way to kind of push or like steer this model a little bit towards your particular use or suite models.

Right.

If I'm jumping ahead to something that you're going to cover, tell me and I'll back off.

But you mentioned accuracy a couple of times accuracy, obviously a big, big thing.

Do the reasoning models themselves improve accuracy?

And I know it's not like there's one kind of reasoning model that fits everything, but I'm just wondering, as you've mentioned it, like, is that part of what the reasoning model is meant to do or is it just kind of a happy side effect or, um, you know, it can in a, in a system, right.

And certainly if you look at reasoning models, there's, you know, if we talk about Nvidia's model, right?

Like mom and Neematran reason super, I might've messed up.

Neematran super with reasoning.

Yes.

Uh, exactly.

If you look at that model specifically, it does have a higher accuracy than these other reasoning models, right?

Like in its, its kind of system where I think, or where we see this being advantageous is as part of a, a larger agentic system where I've got this reasoning model and there are good tasks for reasoning models.

And I'm just going to say that they're bad tasks for reasoning models, right?

Like just like there's good uses for a fork and there's really bad uses for a fork, right?

Like it's the same for this reasoning model.

And with our reasoning model with Lamini Neematran reason, one of the great things about it is that you can turn reasoning on or off.

So you've got a single model that now operates as a reasoning model, or it can operate as just a regular, like non reasoning model.

And so we talk about accuracy.

One of the really nice things about reasoning models is the ability to iterate with them.

I'm giving an example.

You're in a deep researcher type environment and you're like, Hey, this is the topic I want.

I might upload it a few PDFs, right?

Like I give it the structure.

I want to have an intro, a, you know, a conclusion, right?

Keep it pretty light.

You know, you give it a listen.

A model can go, a reasoning model can go and say, okay, I'm going to really quickly generate a lot of, a lot of ideas, a lot of questions that I want to do.

I'm going to present it to you human.

It's like, this is what I'm going to do.

This is like seven tasks or something like that.

Something the human can, is sticky for the human in their brain, right?

Like not a hundred.

And the human can go through and they can be like, Oh, that one looks good.

Can you slightly that one, I want you to re tweak it, right?

I wanted about this topic or not that topic.

That one looks like it's going to be too high level.

Can you take that more towards a 12th grade level?

Right.

And it's really in my mind, it's the improved accuracy that we do see a little bit in the reasoning models, but it's the human model connection that lets you really, it becomes a coworker.

These agentic system become the coworker where you can guide them and you can fine tune them and you can talk with them and interact with them.

So you have the report that you want, and then you do all of the deep researching.

Now you're in minutes of like, I'm going to do this stuff.

And what if I take us, I'm going to date myself.

I'm going to take us way back.

If you remember the internet, when we used to talk about things like load times and you know what I mean?

Like what's the, what's the user's perceived load time of this?

And we would, we would do things to do that.

We now have this ability with the reasoning model, you can interact with it early.

And so when you get to the final report or a draft report, you haven't spent five minutes and it's something you totally different want.

Right.

Didn't want.

Yeah.

So not only is the actual measured accuracy up, but the perceived accuracy from the human, the perceived efficiency, because you were involved in the process early on.

Yeah.

Fantastic.

I feel like that's a great segue into talking about enterprise, the enterprise, IT leaders, whoever's designing, developing the applications.

What are some of the things that should be considered by, you know, a tech leader, a CIO when they're designing an agentic AI system in particular?

We have a couple hours, right?

Yeah.

So I think the biggest one, right, is that when I talk with, with anybody right in the enterprise, I remind them, I'll be like, okay, so what is the agentic system we should use?

And I'm like, I'm going to answer your question, but let me ask you a parallel question.

I'm like, tell me what is the, you know, tell me, what's the one piece of software you use in your IT system?

And they're like, what do you mean one piece of software?

We have 15 different vendors and we do all this kind of stuff.

I'm like, correct.

Right.

I'm like, so you have to think about agentic systems as the same, right?

You're going to get some from your vendors, right?

Whoever that is, you're going to have an application, you're going to have your CRM, you're going to have all these different things for your developers.

They're going to put their own agents in that, right?

And you're going to work with those.

Some are going to be homegrown by you, right?

Because everyone's moving really fast in this space.

You have your own enterprise data with your own sources.

If I can, if I can talk about a very specific Nvidia example that I tell everyone, we have this thing called NV bugs pro AI search.

And I like it for two reasons.

One, it uses AI agents and two, the name just rolls right off the tongue.

Totally.

Right.

At least when you say it, it's in the bugs pro AI search.

It's just like that one sticks in my brain.

Not the order of super or reason, right?

But not that one.

But the thing I like about this is the RIT, our great IT department, who are much more engineers than the average IT department, created this and they created it very early on in in lane chains existence.

And then they modified a little bit.

And now we're here fast forward six months.

And we want to not only use it with in the bugs pro, but we want to hook it up to coding repos and forums and CRM systems and all of that.

And what that means is that now you're in this similar situation where I might have various different vendors and stuff that I've grown myself.

So when I talk with them and say what you're really looking for is you need to look at it in that same context.

It's not I'm going to buy one piece of software, I'm going to write one piece of software, you're gonna have all these agents working together.

And the trick is, how do you let them all come together, mesh together in a somewhat seamless way for employees.

So when I log into our systems, or if you log into our systems, it's context aware, it gives me the information that I need, it helps me do my job, I look at it like that.

It's a look at it in the in the traditional IT deployment sense.

Right, right.

How does that differ from what enterprise IT departments and developers were doing, you know, sort of pre LLM, pre-agenda care, that kind of thing?

Is it to me being a little bit on the outside thinking about it, it sounds sort of like, oh, you've got different vendors, and it's like an app store and CRM systems have their own apps, everything's got their own like, you know, place where you can go grab apps.

Are we moving towards that for agents?

You know, if he had been, I think so.

Yeah, I mean, there definitely will be.

I know they're out there already.

Yeah, right.

You can everyone has their own.

And, and that's fine.

I think the difference is, I say, you know, when we had all these apps that in the, I guess, in the before times, I guess, when we had all these apps, we were in data silos.

Yeah.

And now it's very, it could be possible.

We're in these agentic silos, right, like a little bit where I have data here and I interact with that agent to talk with us.

The big difference though, is we're in a situation now where we're not always going to have API to API access, right?

I don't have to necessarily always have a developer script and code that I can have CRM agent over here talking to confluence wiki agent over here.

And they are communicating, not via an API that a developer set up, they're communicating via human language, right?

Like with each other.

Right.

And that kind of makes connection, it kind of eases the connection points.

It raises some, some things you have to look at, right?

Like, in how are we monitoring this systems?

How are we observing these systems?

Right.

But that is one of the key differences is you're going to not just have API to API to API specs, you're going to have some of that.

And then you're going to have these agents that are just communicating with themselves on your behalf.

Right.

What kind of complexities does that raise or perhaps doesn't raise when it comes to data security?

And obviously, this all runs on data, like you said at the beginning, right?

And just the data.

So if you've got different agents and they're maybe from different vendors and they're doing, how do they talk to each other in a secure manner?

It's not a hundred percent solved yet, right?

Like, but where it's headed is this idea that we call context-based security.

And so if you go back from the history and you're like, you know, security really started with, let's say, firewalls, right?

And what was the motion?

I'm going to put everything in this circle and I'm going to put my hands around it.

And then I secured it.

And then we did this thing where we, I don't know if you remember it, we moved to the cloud, or I think we did, right?

Like, we moved to the cloud and then it was like, oh, oh, I can't put my hands around it anymore.

Now I'm application-based security, right?

Like, because I have pieces everywhere.

And now we're in this motion where every, there's cloud, there's on-prem, we solve all of that.

But when I am accessing a system, like I said, versus maybe with like our CFO is accessing that same system, we're looking at it in a different context, right?

And the information went out of it.

So we're moving to this context-based type of security, where you not only have to understand the person and the credentials and do all that stuff that you are already doing or supposedly doing, right?

Like, our back should be there, right?

Like, if it's not there, we have a more basic problem.

But you have to look at the context in which the question is being asked.

Like, look at the things around it, look at what pieces of information are coming with it.

Do pieces of security before that question and then do an analysis before you return the information to the user.

And you can look at that context.

So it does, I don't want to say complicates it, but it does add about maybe 10% new stuff that we weren't doing before to roughly 90% of just like regular security applications.

I was speaking with Bartley Richardson.

Bartley is the Senior Director of Engineering and AI Infrastructure at NVIDIA.

And we've been talking to agents at Gentec AI, a little bit of cyber security got snuck in there.

But that's Bradley's domain.

He leads a Gentec AI and cyber security AI here at NVIDIA.

And we're talking about a Gentec AI in the enterprise.

To shift gears a little bit and kind of talk about business needs.

Designing one of these systems, are there best practices or things that you've seen that work well when it comes to making sure that you're designing with the business care abouts in mind and then going back and kind of recheck and keeping that North Star.

Well, I think, yeah, there are some of it is kind of what we've already talked about, right?

Like building on these strong foundations of not just tooling and ingest, but models.

And so there's all of that, right?

Like obviously, one thing that we haven't talked about explicitly yet is this notion of traceability and observability and profiling.

Right.

You mentioned briefly, yeah.

Yeah.

And you have to imagine if we go back and we have this kind of distributed system, there's all these agents, they're connected with other things, they're talking in various modalities faster than a human can understand from different vendors and different even like agentic framework providers, right?

Like you might have someone in your business wrote this thing in Lang chain and someone wrote this thing in crew AI and someone wrote this, you know, there's all these different things.

So how do you have a holistic kind of traceability and observability platform like across that?

And that becomes, it becomes a little challenging and it's why we made this new thing.

It's called agent IQ and it's not an agentic framework.

There's plenty of those.

I haven't checked.

We've been talking for a few minutes.

There probably are 10 more that popped up while we were talking, right?

We don't need another one of those.

Our frameworks, the new prompt engineer.

Yes.

They're the new.

Yeah, exactly.

But it's not, it's not one of those, but rather it's a really simple, as much as necessary, as little as possible way to get all of these frameworks and tools and everything to work together and be observable on the same point, being a CS person and coming from an engineering background, the way I tell people is I'm like, again, we're really good at complicating things.

We got agents and tools and all these things.

I'm like, no, everything's a function call.

And so what agent IQ does is it lets you use the frameworks that you were using and you still develop in those frameworks.

Like I was saying, you probably already have stuff.

Don't rewrite that code.

Right.

Let's just hook it up to other things that you have, but let's do that in a way where we develop everything to a function call.

And that lets you say, oh, I've got this, this agent pipeline here and now I want to add a capability to it.

Well, I can add it on or I can wrap it in something else.

So now I have an agent inside of an agent.

It allows this nesting kind of capability.

And what it gets you is this really cool traceability.

Yeah.

So you can go into every tool call, every LLM call, every chain of tool calls.

You can look at the input tokens, the output prompt output tokens, the time it took to do the tool call, the sequence of actions of the tool call.

Yeah.

And you can and we have customers that are already seeing, you know, they look at the timing charts and they'll optimize their tool calling chains and they get the 15 X, right?

Like speed up through their, through their pipeline for it, right?

Or they'll get a, you know, a five X improvement in accuracy by moving things around.

And it's that type of information that is part, one of the reasons that something like agent exists.

That's fantastic.

We've got a few minutes left.

Is there an area that we haven't covered that you want to go into in particular?

Yeah.

I mean, the thing I think I would, I would in Han, right?

If we're talking about agents is, I feel like as an industry, we've been talking about them for a really long time.

Yeah.

But we're pretty, we're pretty new, right?

Like to them and their actual usability, right?

Like, and, and how we're, how we're kind of getting them into, like we said, enterprise scenarios.

I think what's, what's incredibly interesting is these use cases that we have in some of them we have here in booth demos, right?

At GTC this year, right?

So I'm sure afterwards we'll, we'll have those right where people can consume as well.

But we have this one that is, is automating a lot of what happens in the kind of feature requests.

Like what issues are a customer having with my product all the way through writing a PRD for that a product requirement stock all the way to assigning, you know, who's who might be the best engineers for doing this.

I think the coolest thing that I've seen as part of that is we use a reasoning model to say like, Oh, here's some, here's some issues your customers are talking about in forums or here's people, you know, tickets they filed, right?

Here's a set of brainstorming questions that I reason through, go have a meeting.

And then the humans get in the room, they had the meeting about, they talk about, they might draw some diagrams and that kind of stuff.

That goes back into the system.

So think about this as a big human in the loop that goes back into the system, what comes out the other side, a fully formed PRD where we've taken the language and the action items from the actual team's call.

Yeah, we've taken pictures of the whiteboard, it produces the diagrams and then it gives you a PRD that then, you know, if it gets you 75, 80% of the way there, that's fantastic.

That's great.

Because what's, you know, I'm sure, you know, you do your fair share of writing, right?

Like the hardest part for me about writing is that blank blank page.

Totally.

Right.

And if I can get something that's 80% of the way there, it's great.

And that's the point I would leave on is agentic systems and models will make mistakes, right?

Like they will, they will not never be 100% accurate.

I would challenge you to find anything that's 100% accurate.

Right.

But the way to think about them is like, look, the human will be in the loop.

And if it gets you 60 7080% of the way there.

That's amazing.

Yeah, I mean, that's 70% of work you don't have to do it's there.

Exactly.

And that's the part that I think is incredibly compelling.

And we should focus on accuracy.

And we should always try to make things better.

But I never want to lose sight of like, look at the 70% right like that that we did and how are we going to continue to make it better?

Of course, absolutely.

All right, Bartley, before we let you go, little change of pace, but in your own daily routine, your own daily work, are there AI powered tools that you're using every day that you know, you can recommend them it can just be a category specific tool.

What are you using that you like?

Yeah, I'll preface this by saying I have no official affiliation with any of these are some of them I pay a lot for, right?

But other than that, right?

No official affiliation.

Of course, I think like a lot of people, you know, AI powered search engines, if that's something like a like, like a perplexity, right?

I use that a lot.

I use chat GPT, the reasoning model in chat GPT, like quite a bit like that one's fantastic.

If I'm trying to like research things or if I you know, I just have something that I wanted to think about just a little bit more.

Of course, that was before our reasoning model came out before a IQ, right, is out there, which is our deep researcher.

So those are kind of the obvious ones.

I don't get to code as you know, my teams don't let me code as much as they used to and they shouldn't.

That's smart.

That's smart of them.

But I love cursor as a coding tool.

And even just making diagrams, like it's really good at that the amount of times that I just hit tab and I'm like, wow, get out of my brain, right?

Like I just hit tab.

The other one that I use a lot is called napkin, napkin.ai.

Okay, that's the one that I don't know if a lot of people have heard.

Yeah, I haven't heard that one.

Yeah, it's in beta right now.

It's free.

Again, I have no affiliation with them.

I love their tool.

What you can do is you can either just like type freeform text or you can do like a perplexity style and it'll research things for you.

And it will take it let's say you have a process diagram or a complex tree or whatever.

And it turns that into the associated diagram and the infographic for you.

And then what I love about it is it gives you kind of pre formatted options to start from do you want this to be a funnel?

Is this a process tree?

Is it the cycle and all this in different color options?

You're like, yeah, I want that.

And then you can go in and you can change anything.

You can move the text, you can change the text, you can move the shapes and all that.

So I talked about the 70% there, right?

I would toil away with these in PowerPoint for so long, right?

But now and it's getting better, I can just describe it to napkin back of the napkin.

And it produces this great SVG or PNG diagram.

And so it's fantastic.

Bartley, for folks listening who want to find out more about all the things you've been talking about, all the work you and videos doing with the enterprise, where's a good place to go online or places to get started?

Yeah, a great place to go is build.invidia.com.

Right?

Like, so that's a great place to get started.

You can see all of our models, all of our blueprints, which is just kind of like intent examples.

So if you go to build.invidia.com, ai.invidia.com takes you to a similar place, right?

If you're interested in this new thing that we were just talking about, which is called Agent IQ, we'll play on agentic, right?

It's agent IQ.

Oh, yeah, right.

Right.

Yeah, it was clever.

But if you're interested in that, that is open source software on GitHub.

So intent.com/invidia/agent IQ.

Great.

But we'll also be all linked off of build.invidia.com.

Yeah, great.

Bartley Richardson, thank you so much for taking the time out.

Best of luck this week.

Have a great show.

And in all you're doing, maybe we can catch up and do it again down the line and, you know, see if we're still talking about PDFs in a year or two.

Oh my goodness, Noah.

Yes, we'd love to.

We'd love to.

[Music] [Music]