Thursday, July 30th, 2020 | Written by Renee

Facebook has developed yet another poker-playing AI, this time using a general framework that does not rely heavily on domain knowledge. Recursive Belief-based Learning (ReBel) revolves around game state concepts, operating via two AI models that enable it to create a public belief state, which in turn gives it the capability to outclass human players.

How It Works

Like many other AI systems, ReBel also uses reinforcement learning in order to learn a game as quickly as possible. But unlike its predecessors, ReBel implements new concepts that help it go beyond what is visible and known. It trains two AI models one for value, the other for policy and then generates public belief states, quite similar to how human players decipher whats inside the mind of their opponents.

Thus, ReBel considers not only the available data, such as the cards, bet size, or hand range, it digs deeper into the hidden info during self-play, and then creates a subgame to look into probabilities and all the possible actions from opponents as well as the potential outcome of each hand. ReBel then makes a decision around these aspects.

ReBel differs from DeepMinds AlphaZero as it does not base its decisions on mere assumptions; rather, it also takes into account the pot, chips, as well as the agents belief and policies which help it achieve a certain accuracy threshold when making a decision.

Trials Prove ReBel Performs Better Than Poker Pros

To put ReBels capability to the test, it was made to play against one of the top players of heads-up no-limit holdem, Dong Kim, along with three other highly-skilled players.

The experiment showed that ReBel played faster than its human opponents, and defeated heads-up specialist Kim with an aggregated score of 165 thousandths of a big blind, with an average deviation of 69. ReBel also outclassed Facebooks previous poker-playing AI Libratus which achieved an average score of 147 when it was pitted with the top human players back in 2017.

Also worth-noting is the fact that Libratus only defeated Kim by 29 thousandths of a big blind during their trial match. ReBel was trialed in two-player versions of holdem liars dice, and turn endgame holdem, and the results were equally impressive.

Future Applications

The approach used by ReBel has enabled it to master imperfect-information games, making it a viable reference for developing future universal frameworks involving multi-agent interactions in large settings, such as in the field of negotiations, auctions, cybersecurity, and self-driving trucks and cars.

Since it does not depend much on domain knowledge, its algorithms are more geared towards general use in cases with less pre-determined factors. The only issue right now is the potential for it to be used by players as a sophisticated way to cheat when competing at the tables. Facebook quickly addressed this by saying it wont release the ReBel codebase for poker.

The researchers instead opted to open-source their code for Liars Dice which is flexible and easy to understand, and which can also be used in future research.

For the past few years, AI systems have greatly contributed to cracking different complex games. In 2017, Libratus was developed at Carnegie Mellon University initially for learning poker, but its developers had a goal for it to be used in other key areas that are not related to poker, such as medical planning, cybersecurity, and business negotiations. The AI took on four top poker pros and beat all of them.

In 2019, Facebooks AI Lab developed another poker AI called Pluribus, in partnership with the CMU. When it was trialed, it was able to outclass six human players in a traditional no-limit holdem game, making it become the first bot to outclass humans in a multi-player setting. It was also developed using self-play algorithms. Among its human opponents was no other than six-time World Series of Poker bracelet winner Chris Ferguson.

Continued here:

Facebook AI ReBel Capable of Beating Poker Pros | My Blog - Tight Poker