The Watson Surprise

IBM and CMU have written a computer program called Watson that can play Jeopardy, and when it runs on some expensive but achievable hardware it can win against successful human players. If you don't believe me, here's the video, if you do believe me but find yourself thinking that it's all a bit ho-hum and computerish then I want to tell you why I think it is the biggest thing I've learned lately. Either way, there's a nice twenty-minute introductory talk.

We all know that chess computers can beat any human player other than a total freak. IBM have the best, as it happens, the program that defeated Gary Kasparov. But it doesn't seem to be that important. I don't mind being beaten at chess; I'm so rubbish at chess that I've never bothered with it, but I regard myself as reasonably human nonetheless. And there are programs for Snap, and Go, and Noughts and Crosses, all of which which operate more or less successfully against human players. The usual complaint -- especially from losers -- is that the computer is a dull player: in a solved game, one where every legal game can be enumerated, like draughts, it will just pick the exact best move and never lose. Dull. In an unsolved game like Go, or Chess they play textbook openings and end games, and tend to grope for short-term advantage in the middle. Dull. Definitely not human or (nooo!) intelligent.

Jeopardy isn't that sort of game. It's a TV trivia quiz that's designed to be hard to cheat at. The video shows it better than I can describe, but it rewards language comprehension, general -- sometimes very general -- knowledge, and knowing when you don't know the answer. It's designed for humans. Assuming the video I've linked isn't a total con (IBM has many faults, but not weak integrity) the computer is able to play at a level vastly above the point I could ever reach. Whether it wins or loses is irrelevant, if it can hold its own against any human player at all it's doing amazingly well. I am astonished by the quality of what's been achieved here; in one sense I've been expecting this for 25 years, but I never expected it would be so slick, or that it would handle spoken input, or manage without a semantic net.
It's extracting words from the spoken question. I knew speech recognition has come on but I didn't know it had come that far. I'm assuming they're allowed to train it against the questionmaster's voice otherwise I would have said that it was impossible. [12/2 actually -- it is impossible, and it's not what happens. Watson gets the text of the question when the master finishes reading it.]
It's handling the sense if not the meaning of the question. Jeopardy questions are somewhat stereotyped -- contestants will never be asked whether Christianity or Buddhism is the way to live, the right way forward in economic policy or even how to balance a walking stick on their finger -- but still tough enough to force real people to think, and much too tough for any AI I ever saw before. Add on the breadth of the question domain -- the whole of trivia and general knowledge -- and the challenge becomes outrageous.
It's answering -- correctly -- under time pressure. I couldn't answer some of those questions even if I had all the time I wanted. Watson is doing it without any semantic structure -- it's just breaking out the surface grammar, going a little further -- just far enough -- picking out some likely answers from its texts, and, crucially, setting a confidence level that will tell it whether to press the buzzer.
This answering style puts it so far from a human, but close at the same time. Our knowledge structures certainly have a linguistic element, especially in retrieval, but they go deeper -- much deeper -- sometimes. There's much more to reading a book than learning the meaning of the sentences. So Watson doesn't follow us there, though there is no reason why it couldn't eventually come at least some of the way. But where I think it's very human is in the approach to finding the answers (and understanding the questions as well.)

We're not conscious of all our half-baked ideas, the loser remarks that never get access to the speech centres, or the clumsy movement plans that are all suppressed by our internal censors long before they get near the surface. Our consciousness is the narrative constructed around the successes -- the things we actually do and say and think -- and the dream world underneath is mostly inaccessible. But it's real, (mine is apparently really obvious to the people around me!) and it's been copied, or perhaps mimicked (we'll soon learn if that's a meaningful distinction) in Watson. Many different plans kick off at once. They feed into each other, they mark themselves up and down, and in a few hundred milliseconds they settle on a few best picks, and the highest score wins. And because this is Jeopardy, if none of the plans are good, the machine sits tight and does nothing -- but if it thinks it knows, it presses the button and utters the answer.

I hope I've made myself clear here. I don't think it's answering in the exact way a human would. When Stephen Wolfram says it's quite like a search engine, then he's got a point, but then I also think search engines, struggling as they do with relevance, are trying hard to be like us. I realise it keeps up by trading superficial comprehension against inhumanly fast flat access to its entire text base. The memory on the computer array they use is vastly greater than the storage required for the raw knowledge text, and I suspect a fair slice of that goes in the indexing required to turn words into associative symbols. What I do think is that they have a working model of key human intellectual functions. It's at the interesting level -- symbol processing -- rather than the emulated goop of neural networks. And it addresses things we all respect -- language, understanding, knowledge, swiftness and risk. And, unlike the chess programs, it's not a dead end. It's obvious that Watson scales, it's obvious it operates in an open domain, and it's obvious that it could obtain a much deeper level of understanding than it currently does.

Chatter about this is building and it's never been a secret, though they seem pretty coy about the innards. Looking at the quality of the videos on Youtube and other stuff on IBM Research, it's clear that IBM are taking this very seriously, and I suspect that when Watson plays on network TV on 14 February we will all hear a lot about it. So it's worth thinking about what it's not, and whether it matters.

Essentially, Watson is, at the moment, what it was built to be, which is a question answerer. It's not a general purpose artificial intelligence. It can't control a chemical works, or manage NATO Air Land doctrine on a real battlefield. It certainly can't choose between the Christ and the Buddha -- in fact it can't even make a routine investment choice or any other judgement. It can't express a thought. It doesn't want to hear you moaning about Rupert Murdoch even if that's what it takes to pass a Turing test. It doesn't want anything really, it doesn't know anything exists and it's not even dimly conscious. It's never met anybody and it's certainly not a team player. It's definitely still legal to switch it off. But I can't help thinking that the model IBM have built -- the co-operating, competing, alternative processes -- puts it on the way to all of these things. If the researchers go on to get any decent distance down that route, the world will change, and so will we.

Why Kepler (and Fermi) give me Fear

NASA have a magically sensitive light meter in orbit, pointed permanently at a sliver of the sky in Cygnus. This instrument is called Kepler and its purpose is to search for planets orbiting those stars it can see. And that is what it has been doing: it's found fifteen so far, and it's got hundreds of candidates and years to run. Right now it's finding huge planets in close orbits, because they cause frequent big bobbles in the light curve; as time goes on and the data piles up it will be able to find smaller planets further from their suns: planets like Earth. We don't know how many it will find -- data from Kepler and others suggests that solar systems are typically tighter than ours -- but it will certainly find some. And the geometry of transits means that for every world it finds, it'll miss dozens more.

Up till now, it's been possible to believe that planets were rare, that the universe is overwhelmingly plasma: hot or wildly diffuse or both. It's true that planets are insignificant in the total mass of the universe, but we care about them and Kepler tells us that there's plenty to care about. In all probability, scaling from Kepler, the galaxy has countless thousands of rocky worlds with suns that keep them at or near liquid-water temperatures for billions of years. Life worlds in fact -- not worlds we'd necessarily like very much, but suitable homes for whatever's evolved there.

Sixty years ago, most of this was already known. The astronomers had won the long argument: the stars are billions, not millions, of years old. The physicists conceded once they realised that stellar cores are nuclear fusion furnaces that can hold a star like the sun at a roughly constant heat output for eight or ten billion years. That discovery made it intellectually respectable to discuss billion-year ages for the earth, and the life on it.

And that brings us to Enrico Fermi: the great synthesist, father of nuclear power and asker -- sixty years ago -- of the question that's worried me since I heard it. It's called Fermi's paradox. He estimated the number and age of earthy planets -- planets that should build advanced cultures in a few billion years -- and asked:

• "Where Is Everybody?" 

Does that seem strange? Discovery or contact with extraterrestrials is generally imagined as a rare or impossible event. But Fermi was asking why we didn't see them everywhere we looked. He knew that a technical culture -- like ours -- was already conspicuous on a stellar scale, just a hundred years since we started using electricity. The broadcast stations and air defence radars of the war just finished had sent obviously artificial signals that were already passing the nearest stars. In thirty thousand years, the people of Earth will have announced themselves to every planet in the galaxy. That seems like a long time, but against the age of the stars it's infinitesimal. Given the likely number of worlds -- confirmed by Kepler -- there should be many technical civilisations active in the galaxy at once, and since ours is so young most of them will be more advanced -- more conspicuous -- than us.

We can't really tell what an advanced civilisation would do. Perhaps we would see radio emissions. Perhaps we would see large-scale engineering like ringed stars, or even the glare from the engines required to move from star to star. But we all know that there's nothing. The universe looks wild, natural, at every wavelength, at every scale. There may be life on other worlds, there may even be sentience, but to all appearances there are no engineers.

• So. Why is that? Where ARE the aliens?

There are plenty of answers to Fermi's question, but they can be lumped into just two categories: reasons why technical cultures like ours never arise, or why they always fail:

• The"never arise" list is a ragbag -- dozens of improbable or ad hoc claims: "By chance we are the first;" "Life can never develop without a moon that exactly eclipses the sun;" or there's my favourite: "Eukaryotic cells are highly unlikely to evolve." But we are here, so we know that none of these can be absolute. Explaining our loneliness with one of these, or even the whole list, it boils down to being a matter of chance, a fluke. It may BE so, but we don't know, and I struggle to believe that any of these could be true enough, in a galaxy with the countless potential worlds revealed by Kepler.

• The "always fail" list is just as long but much grimmer, because any or all of the reasons could be true,  up in our future, and not far off either. It's not enough to say "Technical civilisations cannot last." They have to disappear quickly, so there's only one at any time, and they can't leave a mark that we would see. Within a generation or two -- it has to be really soon because there are so many candidate planets -- we stop broadcasting. We never terraform the solar system. We never cross to other stars. We are now seeing the height of human development, and soon war, environmental degradation, philosophical attitudes, energy prices, the galactic empire's pest exterminators, or whatever ambush it is, will knock us so far back that tens of millions of years will pass before Earth (it won't be Us) tries, and fails, again. For my money, the "always fail" list seems much more plausible -- any or all of them could be true.

The "always fail" list is horrifying. We're doomed. The fall may be so close that people alive today may have to handle the consequences, and if it's that fast it will not be pleasant. Having those children may turn out to have been a serious moral error.


If you want hope for the future, you need to find an adequate answer somewhere in the "never arise" list. Because, if that's right, then once we're past those hazards (and we already ARE), there's no telling what we'll do, in a galaxy that we own. It's up to us.


The problem is that the best chance, the most plausible reason to permit us to be an oddity, was "planets are rare." It could have been true, but Kepler results show that it's not. We have to look again, to find something improbable in the long history from the lifeless Earth to our technical civilisation. The origin of life seemed a good choice, but we can date that now, and it seems to go back to just a short time after the formation of liquid water: Unless that's a huge fluke, life isn't improbable at all, and it will arise on every water world. 


That's why I know more than I should about eukaryotes. I'm hoping that the step from simple cells to structured ones that can gang up into organisms was difficult, implausible. It took a good few billion years in our history, and I take that as a sign that it was far from inevitable, but I can't really persuade myself that that's true. Effectively, I'm hoping that we can look at  world after earthy world, each with oceans teeming with bacteria, and nothing but slime on the land.


We'll know soon. Here's another name: James Webb. That's the NASA administrator whose name adorns the successor to the Hubble telescope. When Kepler starts listing earthlike worlds, the huge objective mirror of the Webb telescope will be turned towards them, and its spectroscopes will analyse, photon by photon, light that has glanced off alien oceans and strange leaves. Results from each individual planet will tell us little, but as the numbers pile up, we'll know: bacteria, or plants.


If it's plants, that will tell us that eukaryotes are common, and to my mind a world with eukaryotes will have engineers in time. And that says that the reason we seek, the answer to Fermi, is in the always "fail list." I won't be surprised. The "never arise" reasons all boil down to saying that Earth or the Sun or life here is special in some way. In the whole history of modern astronomy, from Copernicus on, that's never been the way to bet. Kepler's observations have taken us one stage further down that road, and there is extinction at the end.