Source: A Meta Lesson, by Andy Kitchen
I’m going to try and summarise both their positions in a few sentences, but you should definitely read both essays, especially as they are so short.
Rich Sutton (approx.): learning and search always outperform hand-crafted solutions given enough compute.
Rodney Brooks (approx.): No, human ingenuity is actually responsible for progress in AI. We can’t just solve problems by throwing more compute at them.
I think both positions are interesting, important and well supported by evidence. But if you read both essays, you’ll see that these positions are also not mutually exclusive, in fact they can be synthesised. But to accept this interpretation you need to take your view one level ‘up’, so to speak.
Rich Sutton isn’t arguing for wasteful learning and search, he’s calling on us to improve it. He is saying we’ll never be able to go back to hand-written StarCraft bots.
The meta lesson is that the most important thing to improve with search and learning — is learning itself.
RE: A Better Lesson, by Rodney Brooks
I think a better lesson to be learned is that we have to take into account the total cost of any solution, and that so far they have all required substantial amounts of human ingenuity.
RE: The Bitter Lesson, by Rich Sutton
We want AI agents that can discover like we can, not which contain what we have discovered. Building in our discoveries only makes it harder to see how the discovering process can be done.