No More Playing Games: AlphaGo AI to Tackle Some Real World Challenges – Singularity Hub

Humankind lost another important battle with artificial intelligence (AI) last month when AlphaGo beat the worlds leading Go player Kie Je by three games to zero.

AlphaGo is an AI program developed by DeepMind, part of Googles parent company Alphabet. Last year it beat another leading player, Lee Se-dol, by four games to one, but since then AlphaGo has substantially improved.

Kie Je described AlphaGos skill as like a god of Go.

AlphaGo will now retire from playing Go, leaving behind a legacy of games played against itself. Theyve been described by one Go expert as like games from far in the future, which humans will study for years to improve their own play.

Go is an ancient game that essentially pits two playersone playing black pieces the other whitefor dominance on board usually marked with 19 horizontal and 19 vertical lines.

Go is a far more difficult game for computers to play than chess, because the number of possible moves in each position is much larger. This makes searching many moves aheadfeasible for computers in chessvery difficult in Go.

DeepMinds breakthrough was the development of general-purpose learning algorithms that can, in principle, be trained in more societal-relevant domains than Go.

DeepMind says the research team behind AlphaGo is looking to pursue other complex problems, such as finding new cures for diseases, dramatically reducing energy consumption or inventing revolutionary new materials. It adds:

"If AI systems prove they are able to unearth significant new knowledge and strategies in these domains too, the breakthroughs could be truly remarkable. We cant wait to see what comes next."

This does open up many opportunities for the future, but challenges still remain.

AlphaGo combines the two most powerful ideas about learning to emerge from the past few decades: deep learning and reinforcement learning. Remarkably, both were originally inspired by how biological brains learn from experience.

In the human brain, sensory information is processed in a series of layers. For instance, visual information is first transformed in the retina, then in the midbrain, and then through many different areas of the cerebral cortex.

This creates a hierarchy of representations where simple, local features are extracted first, and then more complex, global features are built from these.

The AI equivalent is called deep learning; deep because it involves many layers of processing in simple neuron-like computing units.

But to survive in the world, animals need to not only recognize sensory information, but also act on it. Generations of scientists and psychologists have studied how animals learn to take a series of actions that maximize their reward.

This has led to mathematical theories of reinforcement learning that can now be implemented in AI systems. The most powerful of these is temporal difference learning, which improves actions by maximizing expectation of future reward.

By combining deep learning and reinforcement learning in a series of artificial neural networks, AlphaGo first learned human expert-level play in Go from 30 million moves from human games.

But then it started playing against itself, using the outcome of each game to relentlessly refine its decisions about the best move in each board position. A value network learned to predict the likely outcome given any position, while a policy network learned the best action to take in each situation.

Although it couldnt sample every possible board position, AlphaGos neural networks extracted key ideas about strategies that work well in any position. It is these countless hours of self-play that led to AlphaGos improvement over the past year.

Unfortunately, as yet there is no known way to interrogate the network to directly read out what these key ideas are. Instead, we can only study its games and hope to learn from these.

This is one of the problems with using such neural network algorithms to help make decisions in, for instance, the legal system: they cant explain their reasoning.

We still understand relatively little about how biological brains actually learn, and neuroscience will continue to provide new inspiration for improvements in AI.

Humans can learn to become expert Go players based on far less experience than AlphaGo needed to reach that level, so there is clearly room for further developing the algorithms.

Also, much of AlphaGos power is based on a technique called back-propagation learning that helps it correct errors. But the relationship between this and learning in real brains is still unclear.

The game of Go provided a nicely constrained development platform for optimizing these learning algorithms. But many real-world problems are messier than this and have less opportunity for the equivalent of self-play (for instance self-driving cars).

So, are there problems to which the current algorithms can be fairly immediately applied?

One example may be optimization in controlled industrial settings. Here the goal is often to complete a complex series of tasks while satisfying multiple constraints and minimizing cost.

As long as the possibilities can be accurately simulated, these algorithms can explore and learn from a vastly larger space of outcomes than will ever be possible for humans. Thus DeepMinds bold claims seem likely to be realized, and as the company says, we cant wait to see what comes next.

This article was originally published on The Conversation. Read the original article.

Here is the original post:

No More Playing Games: AlphaGo AI to Tackle Some Real World Challenges - Singularity Hub