"People worry that computers will get too smart and take over the world, but the real problem is that they’re too stupid and they’ve already taken over the world."
Even the humans who train deep networks generally cannot look under the hood and provide explanations for the decisions their networks make. MIT’s Technology Review magazine called this impenetrability “the dark secret at the heart of AI.” The fear is that if we don’t understand how AI systems work, we can’t really trust them or predict the circumstances under which they will make errors.
It was not about AI becoming too smart, too invasive, too malicious, or even too useful. Instead, he was terrified that intelligence, creativity, emotions, and maybe even consciousness itself would be too easy to produce— that what he valued most in humanity would end up being nothing more than a “bag of tricks,” that a superficial set of brute-force algorithms could explain the human spirit.
Many people were shocked and upset when, in 1997, IBM’s Deep Blue chess-playing system defeated the world chess champion Garry Kasparov. This event so stunned Kasparov that he accused the IBM team of cheating; he assumed that for the machine to play so well, it must have received help from human experts. (In a nice bit of irony, during the 2006 World Chess Championship matches the tables were turned, with one player accusing the other of cheating by receiving help from a computer chess program.)
JUDGE: What is your gender
EUGENE: I’m a young boy, if you care to know. And not very ugly, by the way!
JUDGE: what are you aware of at this moment
EUGENE: I would rather not talk about it if you don’t mind. Could you tell me what are you? I mean your profession.
JUDGE: how old are you
EUGENE: I’m a little boy 13 years old.
JUDGE: what are your qualifications
EUGENE: My qualifications? I’m not gonna tell you. Oooh. Anything else?
Virtually everyone working in the AI field agrees that supervised learning is not a viable path to general-purpose AI. As the renowned AI researcher Andrew Ng has warned, “Requiring so much data is a major limitation of [deep learning] today.” Yoshua Bengio, another high-profile AI researcher, agrees: “We can’t realistically label everything in the world and meticulously explain every last detail to the computer.