Trying to get a handle on the progress of artificial intelligence is a daunting task, even for those enmeshed in the AI community. But the latest edition of the AI Index report an annual rundown of machine learning data points now in its third year does a good job confirming what you probably already suspected: the AI world is booming in a range of metrics covering research, education, and technical achievements.

The AI Index covers a lot of ground so much so that its creators, which include institutions like Harvard, Stanford, and OpenAI, have also released two new tools just to sift through the information they sourced from. One tool is for searching AI research papers and the other is for investigating country-level data on research and investment.

Most of the 2019 report basically confirms the continuation of trends weve highlighted in previous years. But to save you from having to trudge through its 290 pages, here are some of the more interesting and pertinent points:

All this is impressive, but one big caveat applies: no matter how fast AI improves, its never going to match the achievements accorded to it by pop culture and hyped headlines. This may seem pedantic or even obvious, but its worth remembering that, while the world of artificial intelligence is booming, AI itself is still limited in some important ways.

The best demonstration of this comes from a timeline of human-level performance milestones featured in the AI Index report; a history of moments when AI has matched or surpassed human-level expertise.

The timeline starts in the 1990s when programs first beat humans at checkers and chess, and accelerates with the recent machine learning boom, listing video games and board games where AI has came, saw, and conquered (Go in 2016, Dota 2 in 2018, etc.). This is mixed with miscellaneous tasks like human-level classification of skin cancer images in 2017 and in Chinese to English translation in 2018. (Many experts would take issue with that last achievement being included at all, and note that AI translation is still way behind humans.)

And while this list is impressive, it shouldnt lead you to believe that AI superintelligence is nigh.

For a start, the majority of these milestones come from defeating humans in video games and board games domains that, because of their clear rules and easy simulation, are particularly amenable to AI training. Such training usually relies on AI agents sinking many lifetimes worth of work into a single game, training hundreds of years in a solar day: a fact that highlights how quickly humans learn compared to computers.

Similarly, each achievements was set in a single domain. With very few exceptions, AI systems trained at one task cant transfer what theyve learned to another. A superhuman StarCraft II bot would lose to a five-year-old playing chess. And while an AI might be able to spot breast cancer tumors as accurately as an oncologist, it cant do the same for lung cancer (let alone write a prescription or deliver a diagnosis). In other words: AI systems are single-use tools, not flexible intelligences that are stand-ins for humans.

But and yes, theres another but that doesnt mean AI isnt incredibly useful. As this report shows, despite the limitations of machine learning, it continues to accelerate in terms of funding, interest, and technical achievements.

When thinking about AI limitations and promises, its good to remember the words of machine learning pioneer Andrew Ng: If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future. Were just beginning to find out what happens when those seconds are added up.

Read the rest here:

AI R&D is booming, but general intelligence is still out of reach - The Verge