Following up on my recent posts on the limitations of AI, the New York Times just published a piece by NYU professor Gary Marcus which underlines similar points.
Artificial Intelligence is colossally hyped these days, but the dirty little secret is that it still has a long, long way to go. Sure, A.I. systems have mastered an array of games, from chess and Go to “Jeopardy” and poker, but the technology continues to struggle in the real world…
Even the trendy technique of “deep learning,” which uses artificial neural networks to discern complex statistical correlations in huge amounts of data, often comes up short…
A.I. systems tend to be passive vessels, dredging through data in search of statistical correlations; humans are active engines for discovering how things work.
To get computers to think like humans, we need a new A.I. paradigm, one that places “top down” and “bottom up” knowledge on equal footing. Bottom-up knowledge is the kind of raw information we get directly from our senses, like patterns of light falling on our retina. Top-down knowledge comprises cognitive models of the world and how it works.
Deep learning is very good at bottom-up knowledge, like discerning which patterns of pixels correspond to golden retrievers as opposed to Labradors. But it is no use when it comes to top-down knowledge. If my daughter sees her reflection in a bowl of water, she knows the image is illusory; she knows she is not actually in the bowl. To a deep-learning system, though, there is no difference between the reflection and the real thing, because the system lacks a theory of the world and how it works. Integrating that sort of knowledge of the world may be the next great hurdle in A.I., a prerequisite to grander projects like using A.I. to advance medicine and scientific understanding.
Marcus’s primary specialty is cognitive psychology rather than artificial intelligence. His emphasis on top-down versus bottom-up approaches sounds a lot like the cognitive psych courses I took in college. He is right that the astounding accomplishments of deep learning have almost all been on the bottom-up empirical side, driven by better data and hardware.
For many decades, the forefront of AI had strong roots in top-down approaches like expert systems, decision trees, etc. Those methods have not kept up with the massive improvments in bottom-up techniques. I speculate that for us to reach Artificial General Intelligence, we need to bring top-down approaches back to AI; the question is whether in coming years top-down approaches will show serendipitous breakthroughs the way bottom-up approaches have since 2005.