Mastering the Game of Poker with Machine Learning

Source: Pixabay No Attribution Required

Poker is one of the few card games which truly tests a players mettle. Poker blends the best tactical and strategic plays, with a focus on the psychological elements of the game. Over the years, poker experts and tech aficionados have been working feverishly on creating the ultimate poker learning machine. Poker machines are incredible upgrades of the complex algorithms and self-learning programs that dominated the scene in high-stakes games of chess. While poker machines comprehensively defeated the best online players in the world one-on-one, they failed in championship online poker tournaments, like 6-max and 9-max games.

Ostensibly, machines were encountering tremendous challenges with multi-player online poker games. That has already started to change. A powerful, machine learning poker construct named Pluribus has effectively defeated a handful of poker professionals in the same game. It's the equivalent of one poker professional playing against multiple independent versions of the AI poker construct. The ramifications of such technology are mind-bending. It harkens to the sci-fi blockbusters like Ex Machine (2014), and I Robot (2004) where artificial intelligence successfully outmaneuvered human intelligence and won, time and again.

When poker machines interact with players one-on-one, the dynamics are markedly different to multi-player interaction. While there is certainly complexity in heads up poker games, the fact that they are a zero-sum contest is important. If one player is winning, the other player is losing and vice versa. For this reason, the one-on-one poker format allows sophisticated computers to map out a strategic blueprint for success. The complexity rapidly increases with each additional player that is added to the game, making it extremely challenging for a computer to operate effectively against human opponents. In online poker games with multiple players competing for the pot, the computing capabilities of poker machines must be fine-tuned to the max. Once again, poker is making waves in our cultural zeitgeist!

Experts in the poker arena readily attest to the difficulties of accounting for all possibilities in next to no time at all. Decisions have to be made on the fly, and machines don't necessarily understand the nuances of human behavior, thought processes, strengths and weaknesses to be able to make all those important calls. Success in poker is measured over the long term. This requires the poker player, or poker playing machine, to know when to hold, when to fold, and when to walk away and live to play another day. It appears that the self-styled poker machine prodigy, Pluribus has done precisely that. With thousands of hands played, this artificial poker playing genius appears to be the proverbial real deal. Pluribus didn't learn how to play by pitting itself against human players; it played against copies of itself, outmaneuvering its own thought processes by way of complete randomness.

The complex processes which go into machine learning involve many sophisticated algorithms, programs, and learning curves. One such system is known as Monte Carlo Counterfactual Regret Minimization. This type of didactic learning system is so powerful that it allows the machine to learn from its mistakes, correct those mistakes, and employ winning strategies the next time around. It evaluates different decisions and the outcomes that are generated as a result of them. Known as abstraction of gameplay, and pooling of possibilities, the Monte Carlo Counterfactual Regret Minimization Strategy is highly effective in delivering winning solutions for poker playing machines. While many poker machines perform according to different habits and patterns, and noticeable trends, Pluribus has built-in modifiers with unique strategies to decrease the predictability of outcomes.

There are interesting examples of computers playing against one another where one computer always takes the same outcome and the other acts in a random fashion. When these computers play against humans, the one acting randomly would be more difficult to beat than the one which acts in the same way which would be beaten every time. This deductive ability of humans serves poker players well, but with no noticeable trends, patterns, or behaviors in place, it truly becomes challenging. Believe it or not, the collaboration between Facebook's AI lab and Carnegie Mellon University has truly created a poker playing machine that is capable of beating the best poker professionals in the world.

News of this successful poker machine spread like wildfire, and especially the news that these virtual playing bots can bluff better than human poker players. Pluribus managed to successfully play over 10,000 hands (in 12 days), against a dozen of the world finest poker pros. In one setting, there were 5 human players, and in another setting, they were 5 copies of the same AI program in addition to 1 human player. No collaboration between the copies of Pluribus was permitted. Even the legendary Chris Ferguson of WSOP fame attested to the difficulty in beating Pluribus. While machines beating humans in competitive games is nothing new, this is a whole new level of complexity and success. Multiplayer poker represents the pinnacle of imperfect information, strategy, and psychology in decision-making processes. For a computer to successfully operate and top human performance is a milestone with celebrating!

Original post:

MASTERING THE GAME OF POKER WITH MACHINE LEARNING: How Machine Learning Impacts the World of Online Poker - The Fan Carpet