Monday, December 26, 2011


Smart Guide to 2012: How best to test machine IQ

A hundred years since the birth of Alan Turing, his famous benchmark for machine intelligence is both too hard and too narrow, but there's another way
My heart sinks as a nonsensical response to my question flashes up on the computer screen. I am one of the judges at the 2011 Loebner prize competition, where computer programs are trying to convince us they are people. The contest is based on the Turing test, the most famous benchmark of machine intelligence. So far it's not going well.
2012 is the centenary of the birth of Alan Turing, the second world war code-breaker who dreamed up the test in 1950 while pondering the notion of a thinking machine, so expect a flurry of competitions in his honour. Bear in mind, though, that the Turing test is a poor gauge for today's AIs. For one thing, the test's demand that a program capture the nuances of human speech makes it too hard. At the same time, it is too narrow: with bots influencing the stock market, landing planes and poised to start driving cars, why focus only on linguistic smarts?
One alternative is a suite of mini Turing tests each designed to evaluate machine intelligence in a specific arena. For example, a newly created visual Turing test assesses a bot's ability to understand the spatial relationships between objects in an image against that of a human.
Others want to stop using humans as the benchmark. Using a universal, mathematical definition of intelligence, it could soon be possible to score people and computers on a scale untainted by human bias. Such universal tests should even be able to spot a bot that is far smarter than a human.

No comments:

Post a Comment