Listen to Episode Here
Show Notes
Stuart Russell is one of AI's true pioneers and has been at the forefront of the field for decades. His expertise and forward thinking have culminated in his newest work, Human Compatible: Artificial Intelligence and the Problem of Control. The book is a cornerstone piece, alongside Superintelligence and Life 3.0, that articulates the civilization-scale problem we face of aligning machine intelligence with human goals and values. Not only is this a further articulation and development of the AI alignment problem, but Stuart also proposes a novel solution which bring us to a better understanding of what it will take to create beneficial machine intelligence.
Topics discussed in this episode include:
- Stuart's intentions in writing the book
- The history of intellectual thought leading up to the control problem
- The problem of control
- Why tool AI won't work
- Messages for different audiences
- Stuart's proposed solution to the control problem
Key points from Stuart:
- "I think it was around 2013 that it really struck me that in fact we'd been thinking about AI the wrong way all together. The way we had set up the whole field was basically kind of a copy of human intelligence in that a human is intelligent, if their actions achieve their goals. And so a machine should be intelligent if its actions achieve its goals. And then of course we have to supply the goals in the form of reward functions or cost functions or logical goals statements. And that works up to a point. It works when machines are stupid. And if you provide the wrong objective, then you can reset them and fix the objective and hope that this time what the machine does is actually beneficial to you. But if machines are more intelligent than humans, then giving them the wrong objective would basically be setting up a kind of a chess match between humanity and a machine that has an objective that's across purposes with our own. And we wouldn't win that chess match."
- "So when a human gives an objective to another human, it's perfectly clear that that's not the sole life mission. So you ask someone to fetch the coffee, that doesn't mean fetch the coffee at all costs. It just means on the whole, I'd rather have coffee than not, but you know, don't kill anyone to get the coffee. Don't empty out my bank account to get the coffee. Don't trudge 300 miles across the desert to get the coffee. In the standard model of AI, the machine doesn't understand any of that. It just takes the objective and that's its sole purpose in life. The more general model would be that the machine understands that the human has internally some overall preference structure of which this particular objective fetch the coffee or take me to the airport is just a little local manifestation. And machine's purpose should be to help the human realize in the best possible way their overall preference structure. If at the moment that happens to include getting a cup of coffee, that's great or taking him to the airport. But it's always in the background of this much larger preference structure that the machine knows and it doesn't fully understand. One way of thinking about is to say that the standard model of AI assumes that the machine has perfect knowledge of the objective and the model I'm proposing assumes that the model has imperfect knowledge of the objective or partial knowledge of the objective. So it's a strictly more general case."
- "The objective is to reorient the field of AI so that in future we build systems using an approach that doesn't present the same risk as the standard model... That's the message I think for the AI community is the first phase our existence maybe should come to an end and we need to move on to this other way of doing things. Because it's the only way that works as machines become more intelligent. We can't afford to stick with the standard model because as I said, systems with the wrong objective could have arbitrarily bad consequences."
Important timestamps:
0:00 Intro
2:10 Intentions and background on the book
4:30 Human intellectual tradition leading up to the problem of control
7:41 Summary of the structure of the book
8:28 The issue with the current formulation of building intelligent machine systems
10:57 Beginnings of a solution
12:54 Might tool AI be of any help here?
16:30 Core message of the book
20:36 How the book is useful for different audiences
26:30 Inferring the preferences of irrational agents
36:30 Why does this all matter?
39:50 What is really at stake?
45:10 Risks and challenges on the path to beneficial AI
54:55 We should consider laws and regulations around AI
01:03:54 How is this book differentiated from those like it?
Works referenced:
Human Compatible: Artificial Intelligence and the Problem of Control
Occam’s razor is insufficient to infer the preferences of irrational agents
Synthesizing a human's preferences into a utility function with Stuart Armstrong
We hope that you will continue to join in the conversations by following us or subscribing to our podcasts on Youtube, Spotify, SoundCloud, iTunes, Google Play, Stitcher, iHeartRadio, or your preferred podcast site/application. You can find all the AI Alignment Podcasts here.
