Demis Hassabis on DeepMind, AlphaFold, and the Path to AGI

Source material

Transcript

down this.

Thank you so much.

Exciting to be here.

Thanks to everyone for coming.

It's great to be here.

We're so honored to have you at our chocolate factory.

Yes, I had just heard about that.

Looking forward to the chocolate afterwards.

Excellent.

Well, Dennis, we're going to jump right in.

We have one of the OGs in every way.

Original thinkers, founders, visionaries and all things AI, a true believer, true scientist in in Dennis.

We're going to spend the beginning of the conversation about the early days.

Then early days of deep mind, and then we'll get into the science and open up the room for some questions.

So let's jump right in.

So Dennis, you are a chess prodigy.

You also were a founder of a gaming company, you're a neuroscientist, and you're the founder of deep mind.

And now leader of a really big consequential company.

Those seem like pretty unrelated pieces, but you've said that there's a common thread.

Can you walk us through that?

There is a common thread, and maybe I made it into a common thread.

So it could be a post hoc, you know, sort of shaping, but I wanted to do AI for a long time.

So I kind of decided it was the most important thing I could possibly, and most interesting I could think I could do in my teenage years.

And then I picked things to study or do that I felt eventually would help me build a company like DeepMind.

So I had that as a plan from about 15, 16 years old.

I had a detour into games because actually that was in the 90s.

That's where all the most cutting-edge technology was being done.

Obviously, not just in AI, but in graphics, especially including hardware.

Of course, the GPUs we all used today, they were designed for graphics engines, and we used the, I mean, I was using the very first GPUs back in the day in the late 90s.

So there was a lot of really cutting-edge technology, and then all the games I made, including all the games I did for BullFrog, but also my own company, Licks or Studios, all involved AI as the main gameplay component.

So probably the most, you know, well, no, game I made was theme park when I was about 17, and that was a simulation of an amusement park, and thousands of little people came into your theme park and played on your rides and decided what to buy from the shops.

So there was a whole kind of economics AI model underneath, and it was one of the first games of its type along with SimCity.

And when I saw it sold, you know, 10 million plus copies, and when I saw how delighted people were to interact with the AI, you know, that was one of the things that made me think about spending my whole career on it.

And then, of course, neuroscience is to get inspiration from how the brain works, and different algorithmic ideas from that, and then just bringing all those different things together for the style of deep mind when the timing we fell was right.

And then, of course, we use games as the early proving ground for our AI ideas.

So we've got a room full of founders here, and you can relate because your founder not just wants a choice.

Take us back to the first time, a Licks or Studios.

What was that like?

It's not the startup that you're most known for, but it was one that you had incredible success with.

How did you read that and what did you do about building?

Well, look, we start.

I started a Licks or Studios straight out of college, and I was lucky enough to work at Bullfrog Productions, which, so those of you know, games, it was a kind of legendary games studio in the early days of the game industry, probably the best one in the UK in Europe.

And I wanted to do something that combined push AI.

So effectively, I was funding AI back in those days through the back door, through games development.

And then push the forefront of that and combine it with cutting edge creativity.

And I think that's still relevant today with the way we do our Blue Sky Research, but maybe the biggest lesson I learned was you want to be five years ahead of your time, not 50 years ahead.

So we tried to do a game called Republic at a Licks or Studios, which simulated a whole country.

And then the idea of the game is you could sort of overthrow, I think there was a dictator in charge of the country in any number of different ways.

And we basically simulated living breeding cities.

And this is bearing in mind, this is like the late 90s on a Pentium.

So we had to get all the graphics and all the AI for million people working on, you know, a home PC at the time.

So it was a little bit ambitious.

And maybe it was too ambitious.

And it caused some issues.

And I took that lesson with me, like you want to be ahead of your time, you don't want to be obviously when it's obvious to everyone, it's too late.

But if you're 50 years ahead, then there's probably no way you can get it to be successful.

All right.

So speaking of not being too far out of your time, it was 2009 and you decided there would be a GI.

Yes.

Maybe it was any, you know, 10 years ahead of time, that time, better than 50 years.

So tell us about again, room full of founders here.

Tell us about oh nine.

How did you convince the first few brilliant talents?

Because you pulled in really high caliber employees, early team members.

How do you convince them to believe in what seemed like total sci-fi at the time?

Well, there was some interesting threads that I think we picked up on.

I think we thought we were five years ahead, but maybe we were more like 10.

But it was, you know, deep learning.

It just been invented by Jeff Hinton and colleagues sort of in academia, but almost no one had really realized it was a big deal.

We really knew a lot about reinforcement learning and we felt there was huge progress to be made by combining those two techniques, which almost had never been mixed together really, certainly not at any kind of anything other than toy problems in the academic side of James.

They were quite too quite siloed parts of AI.

Then we could see the compute that GPUs at the time were going to be really useful.

Of course, we used TPUs now, but the accelerated computing industry was was going to be very helpful.

Then we also felt at the end of my PhD in postdoc and some of the other people I got together were computational neuroscientists that we had enough ideas and principles from the brain that could be useful, including the idea that reinforcement learning care eventually scale to AI.

We felt we had these ingredients and we almost felt like we were a secret because no one, either an academia industry, really believed that any big progress was possible.

In fact, a lot of the people in academia used to roll out that, you know, literally roll their eyes up at us when we were sort of suggested we would work on AI or strong AI.

It was sometimes called at the time because it was like, well, we know this doesn't work.

So, you know, everyone tried it in the 90s and I did my postdoc at MIT, which was the sort of centre point for expert systems and first all the logic, language systems and it seems amazing to think that now, but I was already feeling that was a KXN, but they, you know, that's still how it was done, both in Cambridge and the UK and also in MIT, these big centres of traditional AI.

And it felt like, but actually that convinced me even more that we were onto something because at least if we were going to fail, we would fail in a different way than people had failed, you know, to get to AI in the 90s.

So that felt like it was worth doing no matter what, even if obviously it was research, we didn't know for sure it would be successful, but at least we would we would fail in an original way if it didn't work.

Was there any common sticking point in that early belief?

Was there something that you had to prove either to yourself or to your early followers to get them on board?

Well, it was, I mean, we had put it this way.

I would have been spent my life on AI no matter what had happened.

So as it's turned out, it's gone, you know, it's sort of gone on the absolutely amazing side of the optimistic side of what we thought.

Still actually within what we were predicting in 2010, we thought it was be a 20-year mission, and I think we're basically exactly on track as a field for that, and obviously we played our part in that.

But even if it hadn't transpired that way and it was still now, niche subject, that's what I would still be doing, because I felt it was the most important technology ever, if it was obvious to me, you know, our original mission statement at DeepMine was step one, solve, solve intelligence, I built AGI, step two, use it to solve everything else.

So it was always, I always thought it was the most important technology that could ever be invented, but also the most interesting one.

So as a tool for science, as an interesting artifact in itself, and actually as one of the best ways to understand our own minds, you know, like the nature of consciousness, dreaming, creativity, all of these questions I have as a neuroscientist, I felt one of the things that was missing was an analysis tool, like AGI, but also a comparison, a country for that you could do sort of a controlled experiment, study, and compare two different systems against each other.

Let's talk about AI for science.

You've been early to that, you've been a believer, and you've been really purist about this.

This is the driving mission.

What about the way you set up DeepMine and set the culture has positioned it to be on the constant forefront of AI for science?

Well, that was the ultimate goal, at least for me, my personal passion is what there's my own drive to build AGI, which is to advance science in medicine and understanding of the world.

It's my expression of that mission was to sort of do it in a meta-way, build the ultimate tool, and then come back when that was ready and use it to make breakthroughs in science.

Things like Alpha Fold that we've done, and I think many more things.

So we've always had that at the heart of what we've been trying to do at DeepMine.

So actually, we've had an AI for science group division led by Pushmee Kole that has existed for nearly a decade now.

Actually, pretty much the day after we got back from Seoul and the Alpha Go match, which is sort of 10 years to the month now is when we started formally started the AI for science efforts, because I was waiting for the algorithms to be powerful enough, and the ideas to be general enough.

And for me, cracking go was that point, that time, that we thought, okay, now we're ready to really apply these ideas to important real-world problems, starting with these big scientific challenges.

So we've always had that in mind as the the most beneficial use of AI, like what could be better than using it to cure diseases, and give us healthier life spans and to help with medicine.

Followed obviously by other really important areas like material science and the environment and energy, and these kinds of topics, which I think AI is also going to be have a huge part to play in the next few years.

And how does AI break through in biology?

You're deeply involved with this, amorphic.

This is an area of deep passion.

You have been a purist on the potential of AI to cure diseases from the very beginning.

When do we have the type of moment that we've had in language and coding, but in biology?

Yeah, well, I mean, I'd argue we've already had one of those moments without default.

So, you know, it's a 50-year grand challenge, protein-folding, and the 3D structure of proteins is incredibly important thing to know about if you want to design medicines or if you want to understand biology.

Of course, it's only one part of the drug discovery process.

It's an important part, but it's only one part.

So, I assume more thick labs, which is our, you know, later spin out, having a lot of fun running that as well.

It is to build adjacent technologies, in more biochemistry and chemistry space, they can actually design the compounds automatically to kind of fit and bind to the right part of the protein.

So, we know, the protein, the shape of the protein, we know that what's on the surface of the protein, and what we have to target.

But now we've got to build the right compounds that, of course, bind strongly to where you want it to bind on the target of interest, but doesn't bind to anything else, I knew it, because that would be a toxic side effect.

So, the dream is to do almost all the exploration, which is 99% of the of the work and the time in silico, and then save the, the, the, the wet lab step, just for the validation step.

So, that would be, you know, I think if we can do that, and I think we can get there in the next few years, I think we could reduce drug discovery times instead of doubt for taking like, you know, an average of 10 years down to months, maybe even weeks, perhaps even days one day.

And then I think then all disease could be in reach, and I think things like personalised medicine will become possible, you know, like personalised variations off of base medicines.

So, I think the whole of the whole medical area drug discovery areas is going to be revolutionised in the next, in the next few years, brilliant.

You talked a lot about AI for science.

Do you think that at some point AI will create new sciences?

All our industrial revolution and thermodynamics will there be something net new, taught fundamentally in our education system, and if so, what would it be like?

Well, I think there's several things that along those lines that I think is going to happen.

So, first of all, the understanding and the analysis of AI systems themselves, I think is going to become a whole science, a kind of engineering science, these are incredible, incredibly interesting artifacts that we are building, and they're incredibly complex as well, as complex, eventually there'll be as complex as the human mind and the brain.

And so, there'll need to be studied.

So, we can understand fully way beyond where we did we are today, how these systems work.

I think there's a whole kind of field, mech interp, is part of that, but there's a lot more I think that we can do to analyze these systems.

So, that'll be a science.

But I think also AI itself will maybe unlock new sciences, which may be what you're getting at.

The one I'm particularly excited about is AI for simulations.

So, I love simulations, or all the games I wrote, not only had AI, but they were simulations, and I think simulations is the way we can address some of the, what we may be think of social sciences, like economics, and other more humanistic subjects, because it's very difficult to do control studies in, you know, why aren't they just sciences like physics today?

Because the problem is there are emergent systems, just like biology actually, and it's very hard to do repeated control experiments.

You know, if you raise interest rates by half a percent, you have to do it in the real world and then see what happens.

You can have theories, you can't run it thousands of times.

But if you could simulate things, really accurately, there may be, there's sort of new sciences to be done where you can sort of rigorously sample from a very accurate simulator.

And then I think that will turn that give, allow us to make much better decisions in these today, what are very uncertain domains?

Well, we'll take to get to those extremely accurate simulations.

World models were kind of sciences necessary and engineering.

Yeah, well, look, I mean, I'm thinking a lot about that in, we do a ton of that work, like learning simulators, basically, wouldn't be.

So, you know, these are in domains where you can't, we don't know the mathematics of it well enough, or it's perhaps too complex, we can't just write, we can't just write directly down a special case simulator.

It's just not accurate enough, doesn't capture all the variables.

We're doing that.

We've done it in with weather.

We have the most accurate kind of weather simulator in the world, weather next, and it's far faster than what can be chosen you whether you're, we can't really, we can't know.

And I'm not sure that would be a good idea.

But the first step is to understand it better.

And but then even biology, you know, we're working on a kind of what I call a virtual cell.

So, yeah, hugely dynamical emergence system.

And I think biology is, is perfect, sort of, machine learning is perfect, description, language, for biology in the same way, math is for physics.

Because I think in biology, and a lot of these natural systems, you have loads of weak signals, weak correlations, tons of data, far too much that any, that human mind can analyse.

But there are connections and correlations and interesting causalities within that, mass of data.

So, I think it's sort of, it's always struck me that machine learning is the perfect tool to describe those kinds of systems, where until today, you know, mathematics hasn't been able to do that, either because we can't manage it as top mathematicians, because two complex or the expressive power of math is not enough for to understand these sort of highly emergent dynamical systems.

Is it also because of the messiness and stochastic nature?

Yeah, sure.

And, and, I mean, eventually, you could, by the way, once you learn these simulators, it may be there's another branch of new branch of science, you could maybe extract some equations from that, once you have the simulator.

So, you have this sort of implicit simulator, intuitive simulator, and then maybe you could extract explicit equations from that, because you are partly, because you could also sample it as many times as you want.

It's fundamental as Maxwell's or maybe, if, I don't know, that exists for such emergent systems, but if they do exist, I don't see why we weren't able to find them in this, with this, with this methods.

That would be amazing.

You've talked about this theory that the basic building walk of, of everything in the universe could be information like, it's a more a theoretical.

How do you think about it that, and what does that mean for traditional classical touring computer?

Well, look, I think you can, of course, all the famous, you know, equals MC squared and all the stuff Einstein did, an energy and matter a kind of equivalent, but I actually think information has a kind of equivalency in the same way.

So, you can think of, you know, the organization of matter and structure, and especially things like biology, they're a resisting entropy, as basically information processing systems at their heart.

So, I think one can convert all of those three kind of quantities into each other, but I have this feeling information is most fundamental.

So, there's a little bit, the opposite way round to the classic physicist thought in the 1920s and things where, you know, it's sort of energy and matter primary.

I actually think it's a better way to understand the world, the universe is to think about it as information first.

And if that's true, and I think there's quite a lot of evidence for that, then of course, AI is even more sort of profound in a sense.

Totally.

Then we think, and it's already pretty profound, because it's also about organizing information and understanding information and constructing information or objects.

So, AI, in my opinion, is all about information processing.

So, I think there's something sort of very deeply connected with these different areas if you think of it through the lens of information processing as the primary way to think about it.

And do you think a classical term machine will be able to compute everything?

Well, I sometimes think, you know, I sometimes sort of think about what we're doing and refer to ourselves as Churring's champion, because Churring machines, I think Alan Churring's one of my all-time, you know, favourite scientific heroes, I think what he did obviously laid the foundations for the computers for computer science, but also AI.

And I think it's one of the most profound results ever is the Churring machine result.

You know, everything that is computable can be computed by relatively simple description of a machine.

So, I think our brains are likely to be approximate Churring machines, and I think it's interesting to think about the connection between Churring machines and quantum computers and quantum systems, but I think at least what we've shown with things like AlphaGo and especially AlphaFold is the classical Churring machine, obviously in the guise of a modern neural network, it can model what was thought to be in the case of protein folding, it's a quantum system, you know, it's some sense, it's a very, you know, dealing with very small particles and one might think you'd have to take into account all the quantum effects of the water bonds and all sorts of things, but it turns out you can get to, on the proximal optimal sort of solution on a classical system.

So, it may turn out there are a lot of things that we think are quant that would need a quantum system to model or run might be modulable on a classical system thought about in the right way.

So, you've talked about AI consistently as a tool, like a telescope or a microscope, astralabe through the centuries, but what you think about a machine that can model, almost anything, you know, let's say it can't even model quantum systems, like you pointed out, when does it stop becoming a tool and will that ever happen?

Well, my, my, my, my feeling strong feeling is we should, in this sort of mission to in Journey to Build, AGI, those of us on that Journey, many of people in this room, you know, I feel like it would be best to build a tool first and incredibly intelligent and useful and precise tool and then cross the the next sort of rubicon.

That's already profound enough and has, you know, of course, the tool could start becoming more and more autonomous and agent like that we're all seeing, we're in the midst of that, the agent ear and now, but then there's a further step of, like, you know, does it have agency, is it conscious these sorts of questions, which are also going to be questions we're going to need to address, but I would, I would recommend we do that as a second step, perhaps using the tool in the first step to help us with those next profound questions and ideally, also, we could understand our own brain and minds better and define things like consciousness a lot more precisely than we can today.

Do you have estimations of what that definition of consciousness might look like?

No, I mean, I haven't got much to add beyond that thousands of years of philosophy hasn't said already, but I mean, it's very clear to me that it's obvious some components are going to be needed.

They're probably necessary, but not sufficient, the things like self awareness and, you know, the idea of self and other, some kind of continuity over time.

So, some of these things are clearly needed for anything that might look like consciousness, but I mean, obviously it's an open question as to, as to what the full definition is.

And I've talked to many of the great philosophers about that Daniel Dennett, obviously sadly passed away recently, but we had a long conversation a few years back about this, and I think, you know, one of the issues is how does a system behave?

Does it behave like a conscious system?

So, that's, you know, you could argue some of the AI systems might end up being able to do that as they get close to AI, but then there's still the question of why do you, you know, why do we think each other are conscious?

One is the way we're behaving, we're behaving like conscious beings, but the other thing is, we're running on the same substrate.

So, I think if both those things are true, then it's pass a bonus to imagine, you're experiencing the same thing I'm experiencing, which is why we don't have that debate about, you know, normally about, is, are each other conscious, but I think we'll, obviously, we'll never have the substrate equivalence with an artificial system.

So, there'll always be, I think it'll be hard to completely close that gap, so you can look at it behaviorally, but what about, exponentially?

There are probably some ways to do that post-AGI, but it's a bit of out of scope today, even for AI for science discussion.

Brilliant.

So, we're going to open the room to questions in just a moment, get your questions ready, but you brought up philosophers.

You've mentioned Kant and Spinoza as two as your two favorite philosophers.

Kant is this, you know, deontological, highly duty-driven philosophers.

Spinoza, almost has this deterministic view of the universe.

How do you kind of connect those two beliefs?

And where is your thinking of how the world works?

Well, the reason I like those two, and they stuck out for me, is that I think Kant, when I was doing my PhD in neuroscience, you know, his sort of statements about the mind creates reality, right?

I think that's basically true.

And so, another reason to study the mind, right, and how the brain works.

And I'm interested ultimately in the nature of reality.

So, we have to understand how the mind is interpreting that.

And so, I think that's for me what I took from Kant.

And then Spinoza, it's more about the, you could almost call spiritual dimension of like, well, if you're trying to understand the universe using science, in my case, as the tool, you're sort of understanding some deep mystery about how the universe works, right, in a really kind of deep way.

And that's what I feel I'm, we're doing, and I'm doing, when, you know, I do my science, and we work on AI, and we're building these tools is somehow we're kind of reading the language of the universe.

Beautiful.

What a beautiful way to say what you do every day.

Dennis, scientist, orator, and philosopher.

We will, before we wrap, do a couple of rapid fire questions.

Thank you for finishing this.

He's not seeing these yet.

Over under, on distribution, year of AGI.

Wow.

Or reject premise of the question.

No, 2030, I'll be in pretty consistent about that.

Okay, 2030.

Yeah.

Must read book, poem, or paper for one of your chief AGI.

Oh, wow.

Um, if when we achieve once we achieve it, um, well, my favorite book is the fabric of reality by David Deutsch.

So, I think that's still holds.

I'd hope to answer the questions in that book with with the AGI.

That's my post AGI work.

Brilliant.

Yeah.

Proudest moments to a foreign deep mind.

Uh, wow.

Um, we've been lucky to have a lot.

I mean, probably alpha-fold.

Okay.

Yeah.

Now, a couple of games questions.

If you were engaged in a high-stake strategy game, a certain base strategy game.

Okay.

Uh, this is a polytopia, right, this games.

Uh, and you could select one scientist from history.

We're thinking the, the Einstein's, the Turings, Newton's, who would you select to be on your team?

Uh, on my team on your team.

Oh, gosh.

Probably Von Neumann, I think.

Great.

I mean, yeah, you want a game theorist, I think.

And I think, I think, I think he's the best.

Yeah.

That's really a big a teammate.

Yeah.

All right.

Yeah.

Well, Dennis, you do it all.

Thank you so much for bringing with us.

Please join me.

Thank you.