Watson Wins Jeopardy Contest 674
NicknamesAreStupid writes "The word is in, Watson beats the two best Jeopardy players. Sure, it cost IBM four years and millions of dollars and requires a room full of hardware. In thirty years it will all fit in your pocket and cost $19.99. Resistance is futile; you will be trivialized."
Re:AI Winter (Score:5, Insightful)
Yes, the natural language processing is impressive. But it takes a really huge computer, and it's really nothing more than a bunch of clever software along with a database of trivia.
Watson showed very clearly how deeply it did not "understand" anything about what it was doing, via the nature of the blunders that it did make.
Underwhelming achievement (Score:3, Insightful)
Various media articles have made clear that Watson has no visual or auditory input. Presumably Watson is receiving a direct digital feed of the tournament questions (oops, answers, I forgot this is Jeopardy). That alone gives Watson a huge timing advantage over the human competitors, who must (effectively) perform voice recognition and OCR to process the clues. On top of that, Watson has the computer-controlled ability to buzz in in four milliseconds, again giving it a huge advantage over the humans, and one that has nothing to do with AI.
Buzzer timing and strategy is a highly significant part of the game of Jeopardy. Given its direct digital feed and its internal computer clock, Watson is not playing this part of the game by the same rules as the humans. Thus, it's not fair to say that Watson wins a "Jeopardy" contest -- Watson has a huge unnatural advantage. In effect, Watson is not playing the same game as what we normally call "Jeopardy." A real Jeopardy contestant has to use eyes and ears and hands in addition to brain.
To be clear, I do think Watson is a worthy achievement. But this feeling is overshadowed by my constant annoyance at the media and others who incorrectly label this achievement as somehow winning a game of Jeopardy.
Re:Underwhelming achievement (Score:5, Insightful)
Correction: competitors must perform voice recognition or OCR to process the clues. The clues are displayed and read, and the contestants are free to ignore either form, if they wish. Similarly, Watson could have had a camera trained on the monitor and performed OCR on the clue. But, given that OCR has been done brilliantly by computers for years now, would adding that into the mix have made much difference at all?
Regarding ringing in, the contestants also get a signal indicating when they can do it, but it's visual. It would have been easy enough to add another camera trained on the light, but why bother?
The engineers involved were trying to solve the interesting problems. Delivering input to each contestant in the most convenient form doesn't seem like much of a concession.
Comment removed (Score:5, Insightful)
Re:Underwhelming achievement (Score:5, Insightful)
Actually, you are wrong about the buzz in. It gives the humans an advantage. The reason is that Watson has a mechanical buzzer that it presses. So the only advantage would come from reacting faster. However, the rules state that you can buzz in only after the host has finished reading the clue. If you buzz in earlier then you are penalized by .25sec.
Watson has a computer clock. It never buzzes in early, and it never suffers the .25sec penalty. The humans did suffer this penalty on several occasions.
It's ridiculous beyond belief to claim that the humans had the advantage in buzzing in.
It's just people whining (Score:5, Insightful)
They miss the real point: That a computer could do a level of natural language processing that was impossible before. They get caught up on bitching about how it wasn't "perfectly fair" or the computer "didn't act just like a human." No, it didn't it is a computer and that was never the point. The point was to try and develop a system that could process a natural language question and extract an accurate answer. It does this amazingly well, better than anything before by leaps and bounds.
The choice of Jeopardy as a medium was for two reasons:
1) It is a ready made challenging format. It is something that is not well suited for a computer or designed for it in any way, and there is a lot of data to work with. Made it a good choice as something to work on designing and testing for.
2) It is a good exhibition/publicity chance. It is a way to show off the research, to generate interest in it. It brings it to the masses in a way they can understand. Some abstract talk about a computer in a lab that parses natural language means nothing. This shows a computer doing something pretty impressive against impressive humans. Really drives it home.
Unfortunately people get all whiny and defensive about it because they feel this is somehow an attack on humanity. They want to find ways to justify that it wasn't "really a fair test" to prove to themselves that the computers haven't "won."
That is just missing the point entirely. They never claimed Watson was a perfect human analogue (were that the case they would have gone for a rather different demonstration probably). They claimed it was an amazing data mining and parsing system, and they had a cool way to show that off.
Personally, I think it is just amazing and represents a new stage in computer language processing.
Re:Underwhelming achievement (Score:3, Insightful)
Since the "controversy" in question is some dude on Slashdot shitting on an achievement orders of magnitude bigger than anything he will ever achieve, I sincerely doubt it made their radar.
Kinda missing the point (Score:5, Insightful)
Who cares, if Watson's artificial reflexes gave it a few milliseconds' advantage on the buzzer? Who even cares if it'd take it a second longer to read the clue via OCR? So what if Watson would be 5% faster or 10% slower, if conditions were slightly different? Moore's Law makes that level of difference utterly irrelevant - in 18 months time, Watson will be *100% faster* (or even today, if IBM just threw more hardware at it).
Deep Blue vs Kasparov was fascinating at the time, but is uninteresting now for the same reason. A decent desktop PC can play at that level. And comparing human vs machine play styles is also largely pointless, in the same way that comparing birds and jets is pointless.
The important part, by far, is that Watson parsed the questions, linked the clues and searched for statistically relevant answers in a human-like time. The amazing fact is, it can actually do it *at all*. Now that today's systems can do this sort of language parsing and information retrieval in a "reasonable" time, it will be increasingly trivial for tomorrow's. It is now all but inevitable that we will have Watson-like systems available to the public, in numerous fields, in corporations and on the web, in your PCs and even your game consoles, in a brief handful of years.
Re:AI Winter (Score:5, Insightful)
It shows what Watson does is still at a search engine level.
Contrast with Jennings' incorrect answer: "missing a hand".
Both don't know the answer and are guessing, but they are guessing at different levels.
I suspect in most cases Watson doesn't know the answer and is guessing - it has a lot of raw data and is very very good at sorting and filtering. But it does not have a very good model of the world that the Jeopardy game is about.
Re:AI Winter (Score:4, Insightful)
Watson is still just following a fixed set of rules
So are you. We call it physics.
Re:Could it be? (Score:5, Insightful)
If he is a slashdot user then he reads the comments, but certainly not TFA. Given his Jeopardy record, It is apparent that he DOES in fact RTFA. Therefore, I concur with 90% confidence that Ken Jennings is not a slashdot reader.
Re:AI Winter (Score:5, Insightful)
Ah the forever shifting goalposts of AI.
http://www.dansdata.com/gz107.htm [dansdata.com]
""A machine will never be able to read the written word."
"A machine will never understand speech."
"A machine will never be able to look at something and figure out what 3D shape it is."
"A machine will never drive a car."
"A machine will never play chess."
"A machine will never play chess well."
"A machine will never beat a chess Grandmaster."
"A machine will never beat my favourite chess Grandmaster."
Go back far enough and you can find people making these same sorts of predictions about tasks that seem simple today. Arithmetic, algebra, spell-checking - all were clearly Things Only the Mind of Man (and of a Few Unusually Intelligent Women, Bless 'Em) Could Ever Do."
"But a funny thing always happens, right after a machine does whatever it is that people previously declared a machine would never do. What happens is, that particular act is demoted from the rarefied world of "artificial intelligence", to mere "automation" or "software engineering".
Apparently, you see, when they said "a machine will never be able to spot-weld a car together", they meant to say "a machine will never be aware that it's welding a car together". So all of those production-line robots aren't actually a triumph of artificial intelligence at all, any more than aircraft autopilots or optical character recognition or the square-root button on a calculator - which, after all, merely duplicated a perfectly obvious slide-rule operation - are.
But don't worry. Once someone comes up with a computer that can carry on an intelligent IM chat with you, that'll be proper AI. (And a machine will never do it, of course!)"
Now of course we can cross off "A machine will never be able to beat the champion at jeopardy"
but of course that's trivial really.... and look at the mistakes it made while beating one of the best human players. obviously since it made odd mistakes it isn't really a triumph of AI.
Re:AI Winter (Score:5, Insightful)
Shifting goal post? Uh I'm pretty sure the goal post has always been the Turing Test, and that was set before computers were invented.