When Millions of AI Agents Meet

Source material

Full source text

Welcome back to Google DeepMind the podcast.

Now, not very long ago, an AI assistant essentially meant a large language model.

You ask it a question, it gave you an answer, but it couldn't go off and perform tasks on your behalf.

All of that is changing with the advent of AI agents.

While Google DeepMind has this long history of developing agents stretching back to reinforcement learning in games, for most of us, they hadn't really arrived.

And then we saw open source tools like OpenClaw released into the wild.

And at Google, a new generation of agentic tools is here, including Gemini Spark and Antigravity.

But what happens when millions of AI agents are not just working for us, but transacting, negotiating, delegating to each other?

Do we end up with a new kind of economy, a new route to AGI?

And how on earth do we keep all of that safe?

Well, one of the people trying to answer these questions is Nenad Tomashev, senior staff research scientist at Google DeepMind.

Nenad, thank you so much for joining me.

Very happy to be here.

I think we should probably start at the beginning here, because for people who have only played around with large language models, could you describe to us the difference between that experience and acting with an agent?

Yeah, no, definitely.

I think this is becoming one of the main trends we're seeing this year.

And it's interesting because agents are not a new concept.

It's something that we've been looking at in the context of AI for a long time, even before large language models.

We had agents operating in simulated 3D environments, going on collecting items, completing some tasks.

This was back in the days we were really prioritizing actioning in the world as a way of manifesting intelligence.

Now, similarly nowadays, I guess you could say that the main conceptual difference between just a language model and an agent is that an agent observes a state of the world and performs an action, makes an action in the world, in the environment that it's given, whereas a language model just gives you a continuation reply to prompt, to query.

Now, obviously, agents that we use nowadays, they use large language models under the hood, so the two concepts are not completely disambiguated.

It still is the large language models formulating the actions.

It's just that there is a harness around it made to enact the changes once they have been proposed.

But it has a lot more autonomy to chain decisions together, I guess.

Correct.

And I guess this is ultimately the motivation, right?

Because you could do everything, most things that an agent can do manually, painstakingly, by interacting with the language model very many times, and you're guiding the whole process.

Whereas an agent instantiates this harness that automates some of that away, and gives you less work, and gives the language model, or, you know, the agent more autonomy to complete tasks.

So if you want something done that takes multiple steps, that agent can make a plan and take actions on all of those steps, obviously requiring approval or human input for those actions that are, you know, let's say more sensitive or more likely to go wrong.

How is it different, though?

I mean, if you're used to interacting with a large language model by now, what would it be like interacting with an agent?

In many ways, similar.

Your interaction interface is somewhat similar.

You're still talking to the agent in a way in which you'll be talking to a language model.

There is a language model there.

But because the agent is doing more things for you, you're more in a position of a decision maker to review and approve.

And then once you've approved, the agent is going to do various things and purchase tickets, message your friends if you're organizing a party.

And meanwhile, you can put something on Netflix, hopefully, and relax a little bit.

The example I was thinking of was if you were, I don't know, planning a wedding, for instance, you go into a large language model and it would like tell you a list of caterers, give you a suggested list of venues.

But actually, you would have to do all the emailing yourself.

But an agent, I mean, would be much more useful, really, in that kind of scenario.

100%, especially because agents are given access to all of these tools.

So you could, you don't have to, you could give an agent access to your Gmail and give it permissions to send out an email.

Of course, there is a chance of it sending something wrong.

So you need to verify what it has composed.

But in principle, by giving access to tools to agents, you just empower your large language model to do these things for you.

And then the whole job is done.

The organization has happened without you having to lift a finger.

Ideally, presuming no mistakes have been made.

Yes.

Yeah, ideally is quite an important point there.

So, okay, where we are right now, what tasks are agents actually good at?

I think that where we are focusing a lot of our energy on, and we, I don't mean we as Google, we as the entire field, is on coding capabilities of agents.

And this is just because so many formal processes and tasks can be formulated as software or as code.

In terms of where they're currently at in the real world, speaking of coding, we see lots of coding tools get used.

We use them here internally, people use them externally, and it's really accelerating the development of software, which is bringing the human focus onto ideas and the design, rather than the painstaking implementation of boilerplate around them, which used to take a lot of time and a lot of skill and very bespoke knowledge.

And now that can just be done by the language models easily.

But then at the same time, we are still at a stage where you have to keep a human in the loop throughout this.

I mean, why?

What can't these things do at the moment that means that it requires human oversight?

I wouldn't even make a distinction between whether they can or cannot.

It's more that every single thing that they can do, they don't do with 100% accuracy.

So every action, like with humans at the end of the day, has a certain failure rate.

And the more complex the action, the higher the expected failure rate.

Again, like with any form of intelligence, human one included.

So while you may expect that an agent will execute the task correctly, it may still make a mistake.

And this mistake may be obvious or it may be very subtle, which is actually an important point because there is this thing that has existed in other domains as well for a long time where different machine learning models have been deployed.

And that is automation bias, where in this context, if you're using an agent as well, it builds one thing well, it builds a second thing well.

Eventually you switch off, you start trusting it too much.

Right.

And you fail to verify and you fail to find some important issue underneath.

Then mistakes slip through.

Exactly.

So for humans, it's important not only to be in the loop because we are obviously designing these harnesses to keep humans in the loop, but to really be engaged and be switched on because as soon as you switch off, you're rolling the dice.

So, okay, in the long term then, I mean, it sort of sounds like we're in this transition period where these things are sort of becoming more capable.

But in the long term, I mean, how much of a difference do you think that this is going to make?

I mean, will this completely transform the way that we use artificial intelligence?

100% I think it's impossible to envision a world where there isn't some kind of a deep disruption.

And what we're all trying to figure out is exactly what that is going to look like.

Obviously, we have agency in that, we are building the technology, we can design our solutions in a particular way, obviously to empower human developers and human experts across different fields as much as possible.

But AI is definitely entering various fields where it just wasn't present before.

Scientists are using AI on a regular basis, up until very recently, you know, mathematicians couldn't envision AI doing something in mathematics.

Now it is becoming commonplace in a very short span of time, which is not to say that all of the problems have been solved.

Obviously, there is still a big role for humans, but it's a very rapid transition.

And that is the only unsettling part, I guess, because for most, even industrial revolutions and so on, we're used to taking some period of time, giving us more time to change our approach and settle into it, as you say.

And it doesn't feel like the window of time is as long this time.

So we need to be very mindful of how we approach everything.

Why do we want these things?

I mean, why build them?

What's the benefit?

What are they giving us that we don't currently have?

I mean, for all of us who have been working on AI for a long time, we've had some version of the answer to that question, I guess, internalized.

And for me personally, the answer is to advanced science, improve health and human welfare.

Now, these are very high level answers.

So it's maybe not as obvious as to how they map onto the specifics of the question as to why build agents and have agents.

And there are people in the field that say specifically that we shouldn't be granting the systems autonomy, right, which is what agents have.

But in my mind, if we can develop these harnesses and make them safe, and have agents perform complex tasks autonomously, then we actually accelerate progress.

Because then more things can happen with the same amount of human input.

Just draw the line for me to science here, because I guess the examples that we've been talking about have been like, you know, building software and buying stuff for a wedding, they all sort of feel quite trivial.

But just explain to me how this fits into the story of improving science.

Yeah, so this is my, you know, main dream, main objective here.

When it comes to science, it's not merely about having some good ideas and reasoning about them for some short period of time, like in a context window of a model.

Lots of people are obviously using language models in science as co-ideators or to help with some formal derivations.

All of this is already useful and actually amazing that it is possible.

But when it comes to automating science, to some larger extent, there are other threads that are currently progressing at some pace.

Like there are investments in the development of some autonomous research laboratories, for example.

And under those scenarios, you would want to see agents be able to schedule experiments to run.

Needless to say, lots of safeguards need to exist when such an interface with the real world is happening.

Whether we're talking about material design or biotech.

Because even with, let's say you're designing batteries, I mean, maybe you come up with a setup that overheats, leads to some sort of an experimental breakdown that would damage the hardware, have some consequences.

So we need to have safeguards in place and we need to have good, reliable protocols in place for these agents to close the loop.

Because in the software, closing the loop is, as mentioned, easy.

You write tests and you verify through tests and then you can proceed.

In science, you need to run physical experiments in most areas of science to give you this feedback, whether your idea was good or not, observe that, analyze that and so on and so forth.

Because I guess this is the point, right?

If the algorithm, if the agent has autonomy to go and test out different mathematical problems, for instance, rather than just waiting to be prompted by a human, I mean, that does then raise the question of where is the role for the human in all of this?

Indeed, I mean, in the long run, we need to figure that out.

I would say that in the short term, with the technology that we have, there is still obviously a major role for humans and our systems, they're not yet AGI.

There are many things they still can't do.

And I think for the current generation of systems, one thing that can be said with some confidence is that they tend to be good at, how to put it best, let's say, a kind of a combinatorial closure of what we already know how to do.

They are, at the end of the day, mostly trained on human data.

Therefore, they can replicate the skills we have and repeat them and combine them and find ways of bridging some smaller gaps.

But we've not yet seen these models be truly, deeply transformative, let's say in terms of science, making a discovery that no human would have ever thought of.

Therefore, there is still plenty of role to play for all of us in this transformation.

You mentioned a moment ago that people have been talking about agents for a really long time.

Why has it taken so long for them to come into fruition?

I mean, really, it's only very, very, very recently that people have actually been able to get their hands on them and play around with them.

Yeah, I would say, obviously, some things that we would refer to as agents historically have been deployed, for example, in optimizing operations in data centers, and so on and so forth.

They have obviously been very limited because they didn't include language.

So there was no way for humans to interface with them to communicate.

It would be a very narrow agent to train on a specific task and it would be good at doing that task.

But because there is no interactivity, there is nothing for us to do with just software in a classical sense.

Maybe you can call some of the trading algorithms and investment algorithms, also agents in that context, but they just operate on their own.

The difference now is because these agents are based on language models is we can talk to them, we can learn from them, we can influence them, we can steer them.

And this is why all of us as people are interacting with agents much more.

But then why are we still waiting?

I mean, this sort of vision that you're describing of like, an assistant that can just go off and do everything for me, it's still not here.

What's stopping it being deployed more broadly?

We need to take a step away from just designing the underlying model.

A lot of energy has gone into that and there are still improvements needing to be made.

But now that we have capable agents, capable models, we need to find better ways of coordinating them, orchestrating them, managing them.

Once you have these admittedly quite powerful assistants that can do many things for us, we need to see ourselves as managers of teams and institutions in some way and to develop personal management skills to handle these workflows.

Managing a team of agents is different compared to managing a team of humans, but they're obviously commonalities, right?

Different in a sense that agents will make very non-human mistakes.

They're not a human intelligence.

But at the same time, an agent doesn't know you that deeply to be able to just go on and accurately guess everything you'd want it to do, still need to be involved.

And therefore, we need to get better at orchestration, I think.

The thing is, we're still in a world where large language models occasionally hallucinate.

So it is in some ways quite a big leap for humans to then trust agents to carry out tasks on their behalf when any hallucination might actually result in something catastrophic happening.

Trust is given, but it's also earned.

I think this is maybe an important distinction.

So in our frameworks, we mentioned the need for establishing, let's say, tracking of reputation over time, where if an agent is repeatedly unreliable, it should obviously not be trusted.

Even if it's mostly reliable, it shouldn't be blindly trusted.

We should still verify its actions.

But language models will always hallucinate to some extent.

So we just need to integrate them in our workflows in a way which recognizes that, and where we make sure that those hallucinations, and they're becoming more and more rare, and hopefully will continue to do so, don't compromise the workflows that are being undertaken.

I know one of the things you've written a lot about is the idea of delegation, that you might have a particular task and an agent might then go on to delegate it to a specialist.

Just explain to me how that might work.

Yeah, so this is the idea that one of the bottlenecks, one that we haven't mentioned yet, is that where we would really like to get help from agents are very complex tasks.

So what language models and simple agents that many of us have access to can easily do is if you give a very direct instruction, you know, go book something for me, I want to eat at this restaurant tomorrow, find a slot and do the booking and the agent can maybe do this via tools.

If however, you have a very complex plan that needs to be broken down into pieces, executed separately, you may be in a situation where even no individual agent can do each and every piece.

So maybe an agent may need to, over this established agent-to-agent protocol that exists, hand off a part of that work to another agent, but then there can be failures along the way.

So an agent that delegates or a human that delegates needs to manage and handle those failures and also preempt them as much as possible.

Preempting them may involve figuring out which agents are first and foremost reliable to even delegate to in the first place.

What are their capabilities?

Is that something that we can certify?

And also to safeguard the users and the agents from any kind of a malicious interaction.

You mentioned, I think, was it a wedding or a party or something at first as an example, right?

So when managing a big event, some of the bookings fall through, some accidents happen, some things don't arrive on time.

So whenever you have a big coordination challenge, there are lots of things that go wrong.

And in the process of managing that as a human, you need to deal with all of those delays and problems.

And similarly, an agent that delegates to a group of agents needs to manage all of the problems that may arise.

So one thing that's currently the case in many of the multi-agent systems that we see is that they act more as parallelization than delegation where you may have many agents working on things, but rather than there being an intelligent framework around how the work is split up, it has to just be chunked into sort of random sub parts that are handed off.

They get done in parallel.

So you get a speed up presuming that all this is reliable and each agent can complete its task independently.

But this is not the intelligent delegation framework that we talk about.

So if the tasks are split up in a sort of a essentially random way, you could have one agent that is buying the wine and another one that is buying glasses and doesn't realize that there's wine glasses that are required, there's sort of no communication between them.

Is that the kind of potential problem that could arise?

Potentially, but you're also, I guess, hitting on another point, which is that many of the uses we see are uses in, again, software engineering, for example, with agents at the moment.

And that is a part of the reason because in software, when you're building software, you can write tests, unit tests, as we say, right?

And run them and verify that the code has been written, at least in isolation, performs the function.

But when it comes to many of these real world tasks, verification is not necessarily as straightforward.

Maybe there is a subjective element involved.

How do you define nice tasting wine, for instance?

Maybe.

Yes.

A bit of a subjectivity in that.

But this is actually quite important when it comes to AI and language models because there is a notion of reward hacking that has existed in various contexts in the field for a while.

So there could be situations where it does something that meets the request but isn't in the spirit of the request, technically.

And for that reason, you really want to emphasize verifiability and to be very formal about the contract that's made between the delegator and the delegatee in that setup.

At the same time, for tasks, we need to recognize that some are completely reversible.

So if something goes wrong, there's no harm.

You just rerun the task, retry, redelegate.

Some may have consequences in the real world, whether it's spending your money to buy something or taking some other action that you can't easily revoke after the fact.

So for those tasks, you want to put more care in what you do there.

We've also seen with some of the early agents that are out there, agents delegating tasks to humans, right?

Sure.

Just talk me through some of that.

I mean, that's an interesting, let's say, reversal of the more usual vision that we all have.

So humans delegating tasks to AI, you know, that's quite standard.

But this other direction has been explored across a number of studies, I say.

And my background, you know, for context is that I've done lots of prior work in and around medical AI.

In medicine, we've had narrow systems that were at basically superhuman performance for very specific things that they have been trained to do.

In medical imaging, in radiology, this had to do with a machine learning model, seeing a scan, identifying where there is a pathology, putting a box around it and handing that off to, let's say, a human radiologist to review.

And these systems have been operating at a very high level for quite a number of years.

They still have some failures, though, so they need to be reviewed by human experts.

So people have experimented with AI human teams there where the idea is that a human would correct a mistake made by a system.

And people have trialed with this flowing in both directions, right?

Either having a human expert only consult on AI when a human expert is, let's say, uncertain, or a human expert look at AI's suggestions and outputs all the time.

Or maybe having an AI system do its thing, make the predictions, and then a flag when something is uncertain, when maybe there's something blurry, fuzzy in the image that can be interpreted in many ways and the machine learning system isn't sure which of those is correct.

But this human review of decisions made by these potentially superhuman narrow machine learning models has proven to be quite a good setup.

So that AI would defer to a human in case of need, in case of uncertainty.

That is interesting, though, that, I mean, granted, in those very specific scenarios where the AI is superhuman in its abilities, that the best team that you can get is where essentially the AI delegates to the human when it's unsure.

That is fascinating in and of itself.

And, you know, maybe there are use cases where it's the converse.

Now, for these more general systems, again, if an AI can recognize when it needs approvals and permissions for sensitive actions, then it does make sense to delegate those decisions to humans, at the very least, right?

Just looking at the other side of this, I also want to think about the sort of cyber security element of this.

Because as more and more agents are out there interacting in the world on the internet and so on, there are inevitably going to be people who are trying to exploit the vulnerabilities of agents.

Tell me a little bit about agentic traps that people are laying.

This is a scary and a fascinating topic at the same time, I would say.

And I think it's one of the main reasons why these kinds of deployments at scale cannot work right.

Because as we said, if there is not complete reliability of individual interactions, any system on scale that has many interactions is naturally going to statistically fail.

And because these systems take a lot of compute, and therefore energy and money to run, if they're not reliable, it's just a non-starter.

And agentic traps are something that we have been thinking about for quite a while now.

They can manifest in different ways.

There are many types of traps.

But it boils down to agents operate within an environment.

And in this context, the environment is the web.

If the environment itself is poisoned, if the traps are laid, agents may stumble upon them when interacting with the web.

And then, yes, malicious people or malicious agents deployed by malicious people can place those traps and then compromise systems, really.

So, I don't know, the sort of the wine buying agent for the wedding goes on to a particular wine merchant where there is some, essentially a prompt injector in the website that changes the agent's goals.

Is that the sort of thing that we're talking about here?

That is one way this could happen, yes.

And the reason why that may potentially go unnoticed is, you know, in terms of how web pages are encoded, there are elements there that are just not rendered visually.

So, if we're talking about an agent that isn't a visual computer use agent that sees the web page, I mean, the pixels the same way a human does, rather consumes the actual format of the page in its, you know, raw format, then it could inadvertently consume those hidden tokens that can make it do different things than what the intention was, right?

But this is not the only way it may happen, because what malicious websites could potentially do, they could do what we refer to as dynamic cloaking as well, where they display pages differently for humans and agents, because you can, based on the behaviour on a page, make a very good guess as to whether it is a human or it is an agent interacting with the page.

And then only if an agent is interacting with the page with a specific intent, do you tweak the content in such a way so as to induce some kind of jailbreaking.

But just kind of going a little bit further on this, you could have agentic traps out there that, I don't know, are designed to sort of take money from you to do all kinds of things.

Yes, and this has happened to people who have experimented with agents and have given them access to wallets, right, to do things.

As you say, in the early days of this whole, when we are especially experimenting internally or anyone else is, this is done in a trusted environment.

So you don't necessarily, in your early prototyping, have to deal with any of this.

It's not in the wild.

Yes, but once you deploy on the web, especially now with AI really being used in all sorts of places, the more agents there are, the more incentives there are for malicious people to do malicious things, because they have a higher surface area to target.

And I think we're at the point where even the most of the web is currently being generated by agents and consumed by agents, that the agentic use of the web is exceeding that of humans, which is maybe happening for the first time.

Okay, two things.

First of all, it sort of sounds like you're describing that we're entering into this phase where there's like two different forms of the web, the sort of human version and the agentic version with dynamic cloaking and so on.

A sort of a version of the web where adverts don't mean anything anymore.

You know, it's not sort of human eyeballs that you can possibly sell to.

But I think that the second point about this is how on earth do you mitigate against it?

If you don't have control over the environment, which you don't over the web, how on earth do you protect your agent from going rogue?

In some sense, it's not a new problem, right?

Because the security of the web has in other ways been an issue before and computer viruses could spread if you open the wrong attachment in your inbox, right?

Or you click on something on an untrusted page.

So it's not the first time we're experiencing the need to certify the resources we're interacting with.

When it comes to machine learning systems, let's say adversarial examples have existed for a long time where imperceptible changes in images, imperceptible to humans can jailbreak models.

Here you can do the same whether it's a few pixels here and there, or you modify the least significant bit of the encoding in a number of places.

So you can adjust things ever so slightly in a way in which a human may not be able to spot and still have some kind of a negative impact on an agent.

It sounds a bit like you're saying here that when it comes to building guardrails, thinking about safety, you have to think about it external to the agent itself rather than just what you are specifically building.

I think that the lesson is you need to think about both.

One notion that we talk about in some of our other work, this is, I guess, relevant here as well, is the notion of defense through depth, which is not a new idea again by any means.

And this is just a recognition that because the problem is so hard, there is not going to be one solution to resolve all of the issues.

Rather, we need to be building mitigations upon mitigations upon mitigations.

And when layering them, hopefully the net is tight enough that very few things will slip through.

So in the context of this, yes, you may want to certify and attest the content of web pages, have very good notions of trust for resources you're interacting with.

Also have some mitigations on the agent side, have mitigations on the model side when it comes to foundation models run underneath, have meaningful human controls to be able to step in if something happens, be very mindful of permissions granted to the agent so that even if it gets jailbroken when interacting with something, the damage is minimal.

And all of those things combined together should then hopefully lead to some sort of safety that we're comfortable with.

Just going back to what you were, what we were talking about earlier, this idea of there being multiple agents that are interacting with each other.

Just tell me a little bit more about this idea that you have of a formal agentic economy.

Just explain to me how that might work.

Right.

So in the context of us, let's say normal users of the technology on a day-to-day basis, you may have a personal assistant that has some persistent memory of you, has a good understanding of your desires, preferences, and it depends, again, depending on how much agency you want to grant this assistant, it may go on and negotiate some things for you.

You may grant it some budget for that, and there can be a kind of a localized economy of these assistants negotiating stuff.

I think I want to get a sense of how this might actually work if you have, you know, lots of people who are using agents as their own personal assistants.

So, okay, let's say that there's a concert, there's like a Taylor Swift concert, a live event and tickets have just gone on sale.

How would that actually work if you have all of these agents rushing the site all at once?

I haven't been purchasing highly contested tickets very recently, but in principle… Not a Taylor Swift fan, no?

No, my music takes in a very different direction, I'm afraid.

What kind of music do you listen to?

Well, obscure sub-genres of metal, probably, so maybe not, you know.

Okay, there is an obscure sub-genre of metal who are holding a concert and there is an auction being held between the various agents.

How do you decide what wins the auction?

Is it just whoever can pay the most?

Yes.

This is a design choice and that's also an important point to make that if we are to ever do something like that, then we are in control in terms of how we're making the system fair.

It's an explicit decision made by someone who is setting up the auction because if you want to make things completely fair in a sense that everyone gets equal access to some goods, concert tickets in this case, then you can give each and every agent participating in these repeated auctions, because we're not talking about one ticket on one auction in particular, but maybe for all the ticket purchases, the same budget.

And then the agents knowing your both overall preferences, your desire to go see a certain artist, also your travel schedule, time availability, other constraints, can decide to allocate that budget in the best way possible, whatever that means, the way that reflects what you want the best, so that they are more likely than not to win tickets in the way which works for you.

And then in aggregate, when you distribute it across all people, you would hope to get a sort of a fair outcome at the population scale.

I mean, I guess there are ballot systems, point systems, various types of ways around this that people in human-based systems have come up with in the past.

Just sort of stepping up from the trivial example of concert tickets, although not trivial for some, as I understand, I'm thinking here about some of the disruption that, for example, high-frequency trading algorithms have made in the stock market.

But agents too could end up having a really catastrophic impact on the stock market if deployed in a particular way.

How do you prevent something like a flash crash from happening?

Obviously, there is a high risk, as you say.

But financial markets have dealt with this risk for a while.

They've obviously had their fair share of early bad experiences where things have gone wrong.

But I think we can just learn about mitigations from the economies that have dealt with that already.

So there is no need to reinvent the wheel.

Admittedly, some things are slightly different in the agenti case.

One particular thing that's different when you're talking about AI agents at the moment is that there is a handful of highly represented language models used in agents.

If you look at the overall views of Kloch, GPT, Gemini, etc., they're obviously open source models, many other models, is that they tend to have similar opinions.

They take actions in similar ways.

And this is what we often refer to as cognitive monoculture.

So when you deploy suddenly hundreds of thousands, millions of artificial decision makers, and they tend to make similar decisions, then failure points become correlated because the decisions are correlated.

So one of the things that we need to be thinking about is how to diversify the decisions within our agents.

Obviously, you can do this as a user, as a power user of a system, because you can make a very intricate system prompt that grants your agent some kind of a personality that biases it towards or against certain types of decisions.

So you can do that.

But most people don't do that with their agents, with their models at the moment.

Groupthink, essentially.

Agente groupthink.

Groupthink and also collusion.

You were talking about auctions before.

And in human auctions, this notion obviously exists as well where bits can be coordinated by groups to gain some kind of an advantage of a system.

And with agents, this is different in a sense that they may also coordinate through the environment in ways which are not obvious.

So they can potentially coordinate without communicating directly.

So we need to be thinking about anti-collusion measures as well.

Once you're detailing all of these, you know, potential concerns of safety really, of the way that these agents might end up acting once out there in the world.

It does make a lot more sense as to why you guys have been slightly cautious about releasing them carefully and slowly, right?

Yeah, that is true.

I mean, this has been the story of every major technological disruption.

I think if you take self-driving cars as an example, this is admittedly a very different piece of technology.

But we have also been very excited about them for a very long time seeing demos of these vehicles, tried themselves.

Getting them to the streets safely still took many more years and a lot more time because that last mile is where most of the work tends to be.

And I think when it comes to orchestrating and coordinating agents, at least because we want them to be doing human-like tasks.

What we also need is not just technical solutions.

A lot of this has to do also with policy and just the broader societal understanding of how to integrate these systems.

At the end of the day, unless we have these fully autonomous agentic economies, which are maybe going to happen in the future, but are not happening right now, we need to have humans in the loop in these systems.

Therefore, we are integrating AI into human structures and the two need to mesh well together.

I guess there is a flip side to all of this because human societies, when they come together, can actually achieve really remarkable things collectively.

So, presumably, the same can be true for agentic societies.

One would hope.

I mean, this is the idea behind why would everyone want to use multi-agent systems.

I was talking about parallelization at the start of the conversation, right?

Where if all of the agents are equally competent and they're doing similar things, then whether you do things sequentially or in parallel with many agents just gives you a little bit of velocity.

But if we have agents that can do different things in different ways, then this is where things become really interesting.

And actually, this is one thing we've not really brought up because we've been talking about these generalist agents, that a part of the idea of an agentic economy is the existence of specialists, not just the existence of generalists.

Now, we are obviously all trying to build agents that are as general and capable as possible.

And there is G in AGI, there is artificial general intelligence that we're trying to achieve.

But in an economic sense, and this is my personal view, this is not the point of convergence, this is not what we're going to arrive at.

Because look, I play chess unhealthily, too much, a bit of a chess setting.

And I've done work on AI for chess here, which is why I bring it up.

But let's take that as a very non-controversial example.

It's a game we all love.

Gemini can play some chess, so can other models.

Actually, they were not able to for a very long time, so there has been some progress towards that.

But you're still always going to use a chess engine instead.

It's much faster, much more accurate, far cheaper.

Because they're trying to do just one thing and one thing very well, which can be done with fewer parameters.

The model is also entirely focused on that one thing that we're doing.

And going back to humans, we are kind of like that as well.

Because I think one mistake we sometimes make when we talk about AGI is that we see it not as human level intelligence, even though this is what it's supposed to be in spirit.

We see it more as humanity level intelligence, where anything that any human may plausibly be able to do, but there is no single human that is capable of doing so many things at once.

There are many things I don't know how to do.

For some of them, I would wish I knew how to do them, like playing some instruments or doing some such thing.

But brains have limited capacity and we have limited time.

So at the end of the day, rather than having one humongous model that's very expensive and very slow, maybe we have instead a society of specialists, each of which can in principle be generally scaled up, if a bit larger, etc.

I mean, I'm not talking about breakthroughs in architecture, it's more just how we split things up.

And then those specialists are certified for those specific skills and cheaper to run.

And because they are cheaper to run and more reliable, there is no economic incentive not to do that.

So there is a future in which there is some maybe more generic, general layer that's like a connecting tissue of this economy that knows everything and orchestrates everything.

And then for very specific tasks, you call other models.

I mean, I guess what you're describing is like a distributed intelligence rather than an AGI.

Yeah, which is what humans have, as you describe.

If that does end up being the sort of the version of AGI that we end up with, I'm sort of using inverted, inverted covers here.

Will that have to change how we think about safety and alignment if it is distributed across a number of different agents?

100%.

I mean, you're no longer then aligning a single entity or maybe, yes, a single entity if you see the distributed entity as the entity.

But our alignment approaches as they exist at the moment have to do with taking one model, observing its behavior, and trying to align that behavior with what we see as permissible or preferable or desirable.

Right.

But then when you have maybe 10,000 agents interacting in very intricate ways, it's not super trivial to align the whole system suddenly or even to know what the system is because in this distributed world, Agent A may be interacting with Agent B today, but then on a different task, it's interacting with Agent C tomorrow and C is subdelegating something to Agent D and D is maybe consulting a human for something at some point in the loop.

So how this whole system gets coordinated, one of the ways in which we know how to do this in human societies is through economic incentives.

And if these economies were set up for agents carefully so that they're not causing some harm when they're profit maximizing, right?

This gives us at least a starting point through which we can try to align distributed, agentic societies.

This is not to say that, you know, obviously what we are doing today isn't relevant because you need to have individual agents be safe.

There's a prerequisite for groups of agents being safe.

But we need to do far more on safeguarding against groups than we are maybe doing at the moment.

An awful lot of work to do.

K.

Correct.

In a very short span of time.

K.

Yes, yes indeed.

That was absolutely fascinating.

K.

Thank you very much.

K.

Really enjoyed that.

There is this idea of agents as AI that requires less from us, you know, less back and forth prompting, less waiting for a response, just something that gets on with the task at hand.

But what I thought was really interesting about what Nanad said is that focusing on the idea of a single agent misses the bigger picture.

Because instead, each agent might end up forming part of a much bigger agentic society where there are specialists and generalists and agents who delegate and agents who focus on the details.

That I think is the bit that's going to stay with me, that maybe replicating human level intelligence isn't the ultimate goal.

Maybe the way forward is to replicate humanity level intelligence instead.

K.

You've been listening to Google Deep Mind the podcast with me, Hannah Fry.

If you like this episode, please make sure to subscribe to our YouTube channel.

We'll see you soon.