Artificial intelligence crosses the poker barrier – The Globe and Mail

You have to know when to hold em and when to fold em.

Now a program developed by computer scientist at the University of Alberta can do both and do it much better than an entire cohort of professional poker players.

The achievement marks a new milestone for artificial intelligence (AI) involving deep learning, a style of programming that mimics certain aspects of how human brains acquire expertise. But while the program, dubbed DeepStack, represents a significant step, a rival U.S. team said that the method by which it was tested against humans was insufficient to reveal the true extent of its capabilities.

Games provide an important testbed for artificial intelligence because they offer a well-defined arena where programming approaches can be evaluated and compared. Last year, another deep learning system developed by Google DeepMind managed to beat the world champion at Go, a board game that is fiendishly complex despite its simple rules because of the number of possible decisions a player can make.

Poker specifically the version known as Heads-Up No-Limit Texas Holdem presents a different kind of challenge. Unlike Go or chess, where both players can assess the state of the game simply by looking at the board, poker players must deal with incomplete knowledge because of cards that are hidden from view.

The essence of poker is being able to make decisions when you dont have all of the information that you need, said Michael Bowling, who leads the universitys computer poker research group. Dr. Bowling added that the same kind of reasoning is often required when computers have to solve real-world problems, which makes poker an attractive hurdle for designers of intelligent systems.

The Alberta group has been working for 20 years on programs that try to solve poker. In 2008, it developed an algorithm that could defeat top human players at the heads-up limit version of the game, in which all bets are of fixed size. There are one thousand billion different decision points than can arise in such a game, a numerical challenge that Dr. Bowling compares to checkers. While not trivial, its a game that a computer can be hardwired to win.

In comparison, the no-limit version of the game is astronomically more complicated because players can choose to bet any amount up to the number of chips in their possession. A winning strategy often involves betting high when the opponent believes incorrectly that his or her hand is the stronger one.

In designing DeepStack, Dr. Bowlings team, together with colleagues at the Czech Technical University in Prague, had to create a system that not only understood the strength of its own hand and make an informed guess about its opponents, but also weigh what its opponent might be thinking in order to bluff and conceal its own intentions.

People think of bluffing as this very human, psychological thing, but it pretty much falls out of the mathematics of the game, Dr. Bowling said. He added that DeepStack had to be able to learn how to bluff, otherwise it would be a terrible player.

The team developed a deep-learning system that tried to make the best choice by looking only a few actions ahead, otherwise it would be overwhelmed by the mathematical possibilities. The system was trained using an army of lesser computers who played through a multitude of game scenarios, gradually building up DeepStacks intuition for what to do. An overview of how the program works along with the results of its matchups against human players were published Thursday in the journal Science.

To test the system, the team recruited 33 professional poker players from 17 countries to go toe-to-toe with DeepStack, with an offer of cash prizes up to $5,000 awarded to the top three players. The players were each given four weeks in late 2016 to complete 3,000 games against the program. Only a third of the human players went the full distance. Of those, all but one were beaten by a significant enough margin to rule out luck.

Tuomas Sandholm, who leads the Alberta groups chief competitor at Carnegie Mellon University in Pittsburgh, said that DeepStack featured a new combination of programming methods that made it a potent player.

However, he cited several weaknesses in the way the system was tested, including the fact that the players DeepStack faced were not the worlds best and the prizes they were offered were likely not sufficient to motivate the players to perform at their sharpest. He also said that 3,000 matches would not provide humans with enough experience to learn to adjust and potentially outsmart the program.

Dr. Sandholms team has been working with a different system called Libratus that runs on a supercomputer and does not employ deep learning. but instead uses a trial-and-error approach called reinforcement learning. In a sign of how close the competition has become, last month Libratus beat a team of four top, human Heads-Up No-Limit Texas Holdem players. There are no plans as yet for a tournament that would pit the two systems against each other.

In the meantime, Dr. Bowling said there was plenty of scope to beef up DeepStacks capabilities and also to try it out on different variations of no-limit poker that more closely resemble a human championship game.

Follow Ivan Semeniuk on Twitter: @ivansemeniuk

Read the original:

Artificial intelligence crosses the poker barrier - The Globe and Mail