This was enough to pass the Turing Test, but not enough to convince a large contingent of industry watchers, many of whom claimed the limited life experience, vocabulary and sophistication of an adolescent boy from a foreign country had acted as a smokescreen to mask a wide range of flaws and weak points in the conversation.

Despite the Royal Society declaring Eugene's success an "important landmark", many are now calling for a more credible test of a machine's ability to reason as a human would. While Eugene was able to immitate natural language, it was only mimicking understanding it did not learn from the interaction, nor did it demonstrate problem solving skills.

A monitor shows a conversation between a human participant andEugene

One alternative, put forward by voice and language software provider Nuance Communications, is the Winograd Schema Challenge. Developed by Hector Levesque, professor of computer science at the University of Toronto, the Winograd Schema Challenge aims to provide a more accurate measure of genuine machine intelligence.

Rather than basing the test on the sort of short free-form conversation suggested by the Turing Test, the Winograd Schema Challenge poses a set of multiple-choice questions that have a form where the answers are expected to be fairly obvious to a layperson, but ambiguous for a machine without human-like reasoning or intelligence.

For example, a Winograd Schema Challenge question might ask: The trophy would not fit in the brown suitcase because it was too big. What was too big? Answer 0: the trophy or Answer 1: the suitcase?

A human who answers these questions correctly typically uses his abilities in spatial reasoning, his knowledge about the typical sizes of objects, and other types of commonsense reasoning, to determine the correct answer.

"Where were going now is really to extract the meaning and the intent of what somebody said and put it into the context of the conversation, and then using things like anaphora to be able to have a conversation with somebody knowing that they are not always going to reference the subject," said John West, solutions architect for Nuance.

"Were starting to build those anaphora into systems right now, so what the Winograd Schema is looking at is the phrases and trying to add further intelligence to the understanding of that phrase."

West said that, most of the artificially intelligent systems in use today like Apple's Siri and Microsoft's Cortana are very domain-specific, so the expectation around what they are able to achieve is restricted to that domain.

See the rest here:

Why the Turing test is obsolete