Stephen Baker is the journalist and author who traced IBM’s Watson project from its inception and development all the way through its TV duel with Ken Jennings and Brad Rutter on Jeopardy earlier this year. He’s in Seattle this week as part of a tour for his book, “Final Jeopardy” – speaking on the Microsoft campus on Wednesday and appearing at Third Place Books in Lake Forest Park on Thursday evening.
I caught up with Baker for coffee downtown today to talk about topics including Watson, Jeopardy, and the future of humans vs. machines. We also discussed the unorthodox publishing strategy that was used for the book — which involved releasing a partial e-book prior to the actual Jeopardy competition — and the lessons learned from that process. In addition, Baker pointed out some possible lessons for Microsoft from how IBM went about the Watson project.
Continue reading for edited excerpts from our discussion.
Even though this was a “Grand Challenge,” from reading the book it seemed in some ways like the prelude to a product launch for IBM’s technology consulting business. They want Watson to be infused into their offerings for businesses.
Baker: That’s the hope.
What are the prospects for that actually happening — for Watson to live on in the business world?
Some kind of technology that goes through massive amounts of data, comes to grips with the data, and can reach conclusions that turn into hypotheses is inevitable. We’re going to have those machines. Whether or not Watson is the platform for that kind of machine remains to be seen. Watson is a horribly inefficient product. For this project, they didn’t care. Inefficiency was part of their business plan for this. Massive servers, massive redundancy, ridiculous electrical costs, all the rest. Didn’t matter. If they could win on Jeopardy and they could build this machine they could deal with the efficiency issues later. And so other tech companies — Microsoft, Google, perhaps startups — might be able to come up with technology like this that, in a focused way, can do what Watson does more efficiently.
In the process of writing this book, what did you learn about the relationship of man and machine, and the ability for machine to replicate the biology and psychology of man?
Baker: Well, OK. Replicate. It comes up with a simulation of some kind of psychology and knowledge of humans. It doesn’t understand anything. But what I came away with was what I already knew, but a deeper appreciation. This technology process is inexorable, it’s only going to continue and it’s going to speed up. The people just solving problems, one by one, and refining their algorithms, and making it every day just a little smarter than it was, are going to create, everyday, ever-smarter machines that are going to do so much of our cognitive work for us that we are going to be facing the challenge of what to do with our own brains. IBM doesn’t like to talk about this, but machines like this are going to put thousands of people out of jobs.
You describe the process that Ken Jennings goes through, where he explains that flash in his head, and pushing the button before he even has the answer, because he knows it’s coming. Can machines ever live up to that?
Baker: No, they can’t. The thing is, they can still produce the answer, but they can’t have the miraculous breakthrough that Ken Jennings has when he sees that it’s Troilus and Cressida or whatever comes to mind. The complexity and the richness of the human brain is so far beyond the likes of Watson right now. But Watson — in its plodding, inefficient way — if it can come up with answers, then it’s going to displace us. That’s a real challenge to us because our whole education system, or much of it, is based on answering questions. And answering questions is the easiest thing to test, and the easiest thing to measure.
We tend to raise kids in the areas where computers are going to slaughter them. And the things that are harder to measure are things like interpersonal skills, deep thought, understanding, context and creativity. And those are the areas where humans are going to remain superior for a long time. But how do we educate people towards those areas, and how much of this knowledge do we need to be able to run a good mind? You can’t just outsource all of the knowledge work to a machine. Because the brain is where everything happens. That’s where all the connections are made, and you can’t say, oh, I’m going to look up this fact on Google and that fact on Google and come up with ten facts and then synthesize them intelligently without a bunch of stuff inside.
So if you take Watson to its logical conclusion, what is the role of the human being?
The role of the human being is to develop ever-smarter tools, and to master those tools and go beyond those tools to solve all of our problems and our needs and our pleasures, and all the rest. And basically not have our tools displace us. But we’ve dealt with this with tractors, all kinds of things throughout history. But this one is a big one. It’s coming really quickly.
What are your thoughts on IBM’s prospects to stay relevant in technology, and what role did the Watson project play there?
Baker: I think it was just what they needed. IBM cannot afford to be anywhere but at the top of computing. They can’t afford to be anything but one of the great computing companies on Earth. If they lose that, then they’re just kind of an expensive consultancy without a lot to offer. They need that, and they need to show people that — to get investors interested in them, and customers interested in them, and also talented computer programmers and other scientists.
They need it strategically, so badly, in a way that Google and Apple don’t but I actually think Microsoft does. There aren’t too many companies in the world that can take 25 PhDs and say, we’re working on something big, figure it out and come back in four years with this big thing. There aren’t too many companies that can do that, and Microsoft is one of them.
These people (at IBM) working with this problem, they not only have Watson, but they have everything they learned while they built Watson. And that’s a valuable thing. I don’t pretend to know Microsoft very well, but I think it’s a worthwhile thing to consider.
A Grand Challenge? Not necessarily a product challenge?
A Grand Challenge. I’m sure people would argue, oh, we’re doing that, because we’ve got — and they could list 10 things that they’re doing that are really exciting. But one of the things I’m going to talk about tomorrow at Microsoft is, there’s an academic model for research, and it involves writing papers –
Which they do.
Baker: A lot. And IBM does, too. But one of the things that (IBM’s David) Ferrucci established early on in this project was, no papers. Because he didn’t want people looking at the incremental improvements. And the incremental improvements could make a good paper. “Hey, we’ve got this algorithm, nobody’s done it before, and look, it raised our results a full percentage. And that’s worth analysis, right?” He said, no papers. We’re working on something bigger. It’s a shifting of the focus and it’s threatening to these PhDs because they’re in this community where a lot is measured by how many papers you write. And that’s how you’re seen.
Meanwhile, these IBM guys are involved in something that seems a little like a joke, because it’s Jeopardy, seems like it’s PR, seems like it’s a bit of a circus.
And yet it paid off.
Baker: I think it did. It’s up to other people to decide.
When I was watching the shows, I kept wondering what would have happened if, instead of reading the questions in text, Watson was required to use voice recognition to hear and analyze Alex Trebek’s voice.
Baker: Oh, it would have been disastrous, because Watson needed the three seconds between the time it received the text and the time Alex Trebek finished reading it, to process. And it would need another second or two to go through all of its understandings of Alex Trebek’s sentence to even come up with its understanding before it began its hunt. So it would be hopeless.
But Ken Jennings and Brad Rutter also read, think and buzz. Alex Trebek’s voice is just background noise. They’re only listening to him to say, “When can I press the buzzer?” Because they’ve read the clue. They’re reading just like Watson. Except they have the advantage that they read like human who understand English as a native language, and Watson is struggling with things that humans would never struggle with.
There were times when Watson’s final Jeopardy bets would baffle me.
Baker: OK, let me give you the one. Watson was trained on millions of games and optimized to be able to predict exactly how many dollars it would benefit to risk in every situation. That’s one area where it’s just superior to humans. Machines do numbers. It’s good at that. At the end of the second day, which was the end of the first game, where Watson flubbed the Final Jeopardy on Chicago and Toronto, the airports, it had a daunting lead, and so at that point, the algorithms of Watson look at its lead and say, what are the chances if I bet $20,000 in an attempt to absolutely bury people, what are the chances that I’ll miss that? Well, it knows that it misses 4 out of 10 Final Jeopardies, so why would it ever risk an almost insurmountable lead. So it bet less than $1,ooo. So in the clue that showed Watson at its most clueless, it also showed it at its most brilliant. And naturally it was clueless in context and brilliant at math.
You had an unusual schedule for writing this book — it didn’t follow the traditional publishing process.
Baker: I learned about this Jeopardy project in November of 2009, just as I was preparing to leave BusinessWeek. I pitched the book in January 2010 and I said, the game is going to be in January (2011), the TV show is going to be in February, we have to really accelerate this process and get this book out no later than September. I was thinking like somebody who had written a book and had suffered through the delays of book publishing.
My editor said, forget that, if we do this book we’re going to produce it the day after the competition. And then they figured that much of the excitement of these types of things was the anticipation, and not afterward, so we wanted something to sell before the match, and that’s where we came up with the idea of the partial e-book. That was a hard sell.
It was a hard sell to who?
Baker: It was a hard sell to anyone who had to change what they’ve always done to do it. And so I appreciate the efforts that they made. The people in marketing at Houghton Mifflin had to do their marketing differently. Amazon and Barnes & Noble had to come up with new ways of selling a book.
There’s some learning that went on during this process. If you downloaded the first eleven chapters (on an e-book) and read them, and let’s say you put copious notes in it. If you wanted the last chapter, you had to get basically a new book, and all of your notes are gone. That’s something we’re going to have to deal with in this world we’re entering. You had two web pages — one for the partial e-book and one for the completed book, so you had comments on one that had to be ported manually to the other. Reviews. I lost a bunch of reviews, but Amazon to its credit brought them back.
What did you learn, and how could it be applied to future books along these lines?
Baker: I think it’s a model for book publishing. It goes back to what writers like Dickens were doing in the 19th Century. Selling a serialized experience. You could sell chapters, and basically you have the buyers become subscribers. Your interaction with them doesn’t end when they buy the book. I could continue to write and funnel content to the people who bought the book, and they could buy it for 50 cents, or 75 cents or whatever it is.
It’s like Xbox Live, downloadable content for Gears of War. Hey, here’s the new map pack.
Baker: That’s it. Now the issue is, I don’t own those customers. B&N and Amazon do. So there would be some tussling because the publishers would want to own those relationships. But what you want is those relationships to exist, and the interest in the product to exist.