Some researchers are searching for more meaningful ways to measure artificial intelligence.

We have self-driving cars, knowledgeable digital assistants, and software capable of putting names to faces as well as any expert. Google recently announced that it had developed software capable of learningentirely without human helphow to play several classic Atari computer games with skill far beyond that of even the most callus-thumbed human player.

But do these displays of machine aptitude represent genuine intelligence? For decades artificial-intelligence experts have struggled to find a practical way to answer the question.

AI is an idea so commonplace that few of us bother to interrogate its meaning. If we did, we might discover a problem tucked inside it: defining intelligence is far from straightforward. If the ability to carry out complex arithmetic and algebra is a sign of intellect, then is a digital calculator, in some sense, gifted? If spatial reasoning is part of the story, then is a robot vacuum cleaner thats capable of navigating its way around a building unaided something of a wunderkind?

The most famous effort to measure machine intelligence does not resolve these questions; instead, it obscures them. In his 1950 paper Computing Machinery and Intelligence, published six years before the term artificial intelligence was coined, the British computer scientist Alan Turing considered the capacity of computers to imitate the human intellect. But he discarded the question Can machines think? The act of thinking is, he argued, too difficult to define. Instead, he turned to a black-box definition: if we accept humans as an intelligent species, then anything that exhibits behaviors indistinguishable from human behavior must also be intelligent. Turing also proposed a test, called the imitation game, in which a computer would prove its intelligence by convincing a person, through conversation, that it is also human. The imitation game was a thought experiment, not a formal scientific test. But as artificial intelligence advanced, the idea took on a life of its own, and the so-called Turing test was born.

In the years since, the Turing test has been widely adopted and also widely criticizednot because of flaws in Turings original idea, but because of flaws in its execution. The best-known example is the Loebner Prize, which in 1990 began offering $100,000 for the first computer whose text conversation several judges deemed indistinguishable from that of a human. The Loebner Prize has been derided for allowing entrants to use cheap tricks, like confusing participants with odd diversions, in place of more honest approaches that uphold the spirit of Turings premise.

A chatbot called Eugene Goostman made headlines last June for supposedly passing the Turing test in a contest organized at the University of Reading in the U.K. The software convinced 30 percent of the human judges involved that it was human. But as many AI experts pointed out at the time, and as transcripts of conversations with Goostman show, the chatbot relies on obfuscation and subterfuge rather than the natural back and forth of intelligent conversation.

Heres an excerpt from one exchange, for example:

Scott:Which is bigger, a shoebox or Mount Everest?

Eugene:I cant make a choice right now. I should think it out later. And I forgot to ask you where you are from

See the rest here:

Rewriting the Rules of Turings Imitation Game