IBM and CMU have written a computer program called Watson that can play Jeopardy, and when it runs on some expensive but achievable hardware it can win against successful human players. If you don't believe me, here's the video, if you do believe me but find yourself thinking that it's all a bit ho-hum and computerish then I want to tell you why I think it is the biggest thing I've learned lately. Either way, there's a nice twenty-minute introductory talk.
We all know that chess computers can beat any human player other than a total freak. IBM have the best, as it happens, the program that defeated Gary Kasparov. But it doesn't seem to be that important. I don't mind being beaten at chess; I'm so rubbish at chess that I've never bothered with it, but I regard myself as reasonably human nonetheless. And there are programs for Snap, and Go, and Noughts and Crosses, all of which which operate more or less successfully against human players. The usual complaint -- especially from losers -- is that the computer is a dull player: in a solved game, one where every legal game can be enumerated, like draughts, it will just pick the exact best move and never lose. Dull. In an unsolved game like Go, or Chess they play textbook openings and end games, and tend to grope for short-term advantage in the middle. Dull. Definitely not human or (nooo!) intelligent.
Jeopardy isn't that sort of game. It's a TV trivia quiz that's designed to be hard to cheat at. The video shows it better than I can describe, but it rewards language comprehension, general -- sometimes very general -- knowledge, and knowing when you don't know the answer. It's designed for humans. Assuming the video I've linked isn't a total con (IBM has many faults, but not weak integrity) the computer is able to play at a level vastly above the point I could ever reach. Whether it wins or loses is irrelevant, if it can hold its own against any human player at all it's doing amazingly well. I am astonished by the quality of what's been achieved here; in one sense I've been expecting this for 25 years, but I never expected it would be so slick, or that it would handle spoken input, or manage without a semantic net.
It's extracting words from the spoken question. I knew speech recognition has come on but I didn't know it had come that far. I'm assuming they're allowed to train it against the questionmaster's voice otherwise I would have said that it was impossible. [12/2 actually -- it is impossible, and it's not what happens. Watson gets the text of the question when the master finishes reading it.]
It's handling the sense if not the meaning of the question. Jeopardy questions are somewhat stereotyped -- contestants will never be asked whether Christianity or Buddhism is the way to live, the right way forward in economic policy or even how to balance a walking stick on their finger -- but still tough enough to force real people to think, and much too tough for any AI I ever saw before. Add on the breadth of the question domain -- the whole of trivia and general knowledge -- and the challenge becomes outrageous.
It's answering -- correctly -- under time pressure. I couldn't answer some of those questions even if I had all the time I wanted. Watson is doing it without any semantic structure -- it's just breaking out the surface grammar, going a little further -- just far enough -- picking out some likely answers from its texts, and, crucially, setting a confidence level that will tell it whether to press the buzzer.
This answering style puts it so far from a human, but close at the same time. Our knowledge structures certainly have a linguistic element, especially in retrieval, but they go deeper -- much deeper -- sometimes. There's much more to reading a book than learning the meaning of the sentences. So Watson doesn't follow us there, though there is no reason why it couldn't eventually come at least some of the way. But where I think it's very human is in the approach to finding the answers (and understanding the questions as well.)
We're not conscious of all our half-baked ideas, the loser remarks that never get access to the speech centres, or the clumsy movement plans that are all suppressed by our internal censors long before they get near the surface. Our consciousness is the narrative constructed around the successes -- the things we actually do and say and think -- and the dream world underneath is mostly inaccessible. But it's real, (mine is apparently really obvious to the people around me!) and it's been copied, or perhaps mimicked (we'll soon learn if that's a meaningful distinction) in Watson. Many different plans kick off at once. They feed into each other, they mark themselves up and down, and in a few hundred milliseconds they settle on a few best picks, and the highest score wins. And because this is Jeopardy, if none of the plans are good, the machine sits tight and does nothing -- but if it thinks it knows, it presses the button and utters the answer.
I hope I've made myself clear here. I don't think it's answering in the exact way a human would. When Stephen Wolfram says it's quite like a search engine, then he's got a point, but then I also think search engines, struggling as they do with relevance, are trying hard to be like us. I realise it keeps up by trading superficial comprehension against inhumanly fast flat access to its entire text base. The memory on the computer array they use is vastly greater than the storage required for the raw knowledge text, and I suspect a fair slice of that goes in the indexing required to turn words into associative symbols. What I do think is that they have a working model of key human intellectual functions. It's at the interesting level -- symbol processing -- rather than the emulated goop of neural networks. And it addresses things we all respect -- language, understanding, knowledge, swiftness and risk. And, unlike the chess programs, it's not a dead end. It's obvious that Watson scales, it's obvious it operates in an open domain, and it's obvious that it could obtain a much deeper level of understanding than it currently does.
Chatter about this is building and it's never been a secret, though they seem pretty coy about the innards. Looking at the quality of the videos on Youtube and other stuff on IBM Research, it's clear that IBM are taking this very seriously, and I suspect that when Watson plays on network TV on 14 February we will all hear a lot about it. So it's worth thinking about what it's not, and whether it matters.
Essentially, Watson is, at the moment, what it was built to be, which is a question answerer. It's not a general purpose artificial intelligence. It can't control a chemical works, or manage NATO Air Land doctrine on a real battlefield. It certainly can't choose between the Christ and the Buddha -- in fact it can't even make a routine investment choice or any other judgement. It can't express a thought. It doesn't want to hear you moaning about Rupert Murdoch even if that's what it takes to pass a Turing test. It doesn't want anything really, it doesn't know anything exists and it's not even dimly conscious. It's never met anybody and it's certainly not a team player. It's definitely still legal to switch it off. But I can't help thinking that the model IBM have built -- the co-operating, competing, alternative processes -- puts it on the way to all of these things. If the researchers go on to get any decent distance down that route, the world will change, and so will we.
No comments:
Post a Comment