Tuomas Sandholm, a computer scientist at Carnegie Mellon University, is not a poker player or much of a poker fan, in fact but he is fascinated by the game for much the same reason as the great game theorist John von Neumann before him. Von Neumann, who died in 1957, viewed poker as the perfect model for human decision making, for finding the balance between skill and chance that accompanies our every choice. He saw poker as the ultimate strategic challenge, combining as it does not just the mathematical elements of a game like chess but the uniquely human, psychological angles that are more difficult to model precisely a view shared years later by Sandholm in his research with artificial intelligence.
WHAT I LEFT OUT is a recurring feature in which book authors are invited to share anecdotes and narratives that, for whatever reason, did not make it into their final manuscripts. In this installment, Maria Konnikova shares a story that was left out of The Biggest Bluff: How I Learned to Pay Attention, Master Myself, and Win (Penguin Press).
Poker is the main benchmark and challenge program for games of imperfect information, Sandholm told me on a warm spring afternoon in 2018, when we met in his offices in Pittsburgh. The game, it turns out, has become the gold standard for developing artificial intelligence.
Tall and thin, with wire-frame glasses and neat brow hair framing a friendly face, Sandholm is behind the creation of three computer programs designed to test their mettle against human poker players: Claudico, Libratus, and most recently, Pluribus. (When we met, Libratus was still a toddler and Pluribus didnt yet exist.) The goal isnt to solve poker, as such, but to create algorithms whose decision making prowess in pokers world of imperfect information and stochastic situations situations that are randomly determined and unable to be predicted can then be applied to other stochastic realms, like the military, business, government, cybersecurity, even health care.
While the first program, Claudico, was summarily beaten by human poker players one broke-ass robot, an observer called it Libratus has triumphed in a series of one-on-one, or heads-up, matches against some of the best online players in the United States.
Libratus relies on three main modules. The first involves a basic blueprint strategy for the whole game, allowing it to reach a much faster equilibrium than its predecessor. It includes an algorithm called the Monte Carlo Counterfactual Regret Minimization, which evaluates all future actions to figure out which one would cause the least amount of regret. Regret, of course, is a human emotion. Regret for a computer simply means realizing that an action that wasnt chosen would have yielded a better outcome than one that was. Intuitively, regret represents how much the AI regrets having not chosen that action in the past, says Sandholm. The higher the regret, the higher the chance of choosing that action next time.
Its a useful way of thinking but one that is incredibly difficult for the human mind to implement. We are notoriously bad at anticipating our future emotions. How much will we regret doing something? How much will we regret not doing something else? For us, its an emotionally laden calculus, and we typically fail to apply it in quite the right way. For a computer, its all about the computation of values. What does it regret not doing the most, the thing that would have yielded the highest possible expected value?
The second module is a sub-game solver that takes into account the mistakes the opponent has made so far and accounts for every hand she could possibly have. And finally, there is a self-improver. This is the area where data and machine learning come into play. Its dangerous to try to exploit your opponent it opens you up to the risk that youll get exploited right back, especially if youre a computer program and your opponent is human. So instead of attempting to do that, the self-improver lets the opponents actions inform the areas where the program should focus. That lets the opponents actions tell us where [they] think theyve found holes in our strategy, Sandholm explained. This allows the algorithm to develop a blueprint strategy to patch those holes.
Its a very human-like adaptation, if you think about it. Im not going to try to outmaneuver you head on. Instead, Im going to see how youre trying to outmaneuver me and respond accordingly. Sun-Tzu would surely approve. Watch how youre perceived, not how you perceive yourself because in the end, youre playing against those who are doing the perceiving, and their opinion, right or not, is the only one that matters when you craft your strategy. Overnight, the algorithm patches up its overall approach according to the resulting analysis.
Theres one final thing Libratus is able to do: play in situations with unknown probabilities. Theres a concept in game theory known as the trembling hand: There are branches of the game tree that, under an optimal strategy, one should theoretically never get to; but with some probability, your all-too-human opponents hand trembles, they take a wrong action, and youre suddenly in a totally unmapped part of the game. Before, that would spell disaster for the computer: An unmapped part of the tree means the program no longer knows how to respond. Now, theres a contingency plan.
Of course, no algorithm is perfect. When Libratus is playing poker, its essentially working in a zero-sum environment. It wins, the opponent loses. The opponent wins, it loses. But while some real-life interactions really are zero-sum cyber warfare comes to mind many others are not nearly as straightforward: My win does not necessarily mean your loss. The pie is not fixed, and our interactions may be more positive-sum than not.
Whats more, real-life applications have to contend with something that a poker algorithm does not: the weights that are assigned to different elements of a decision. In poker, this is a simple value-maximizing process. But what is value in the human realm? Sandholm had to contend with this before, when he helped craft the worlds first kidney exchange. Do you want to be more efficient, giving the maximum number of kidneys as quickly as possible or more fair, which may come at a cost to efficiency? Do you want as many lives as possible saved or do some take priority at the cost of reaching more? Is there a preference for the length of the wait until a transplant? Do kids get preference? And on and on. Its essential, Sandholm says, to separate means and the ends. To figure out the ends, a human has to decide what the goal is.
The world will ultimately become a lot safer with the help of algorithms like Libratus, Sandholm told me. I wasnt sure what he meant. The last thing that most people would do is call poker, with its competition, its winners and losers, its quest to gain the maximum edge over your opponent, a haven of safety.
Logic is good, and the AI is much better at strategic reasoning than humans can ever be, he explained. Its taking out irrationality, emotionality. And its fairer. If you have an AI on your side, it can lift non-experts to the level of experts. Nave negotiators will suddenly have a better weapon. We can start to close off the digital divide.
It was an optimistic note to end on a zero-sum, competitive game yielding a more ultimately fair and rational world.
I wanted to learn more, to see if it was really possible that mathematics and algorithms could ultimately be the future of more human, more psychological interactions. And so, later that day, I accompanied Nick Nystrom, the chief scientist of the Pittsburgh Supercomputing Center the place that runs all of Sandholms poker-AI programs to the actual processing center that make undertakings like Libratus possible.
A half-hour drive found us in a parking lot by a large glass building. Id expected something more futuristic, not the same square, corporate glass squares Ive seen countless times before. The inside, however, was more promising. First the security checkpoint. Then the ride in the elevator down, not up, to roughly three stories below ground, where we found ourselves in a maze of corridors with card readers at every juncture to make sure you dont slip through undetected. A red-lit panel formed the final barrier, leading to a small sliver of space between two sets of doors. I could hear a loud hum coming from the far side.
Let me tell you what youre going to see before we walk in, Nystrom told me. Once we get inside, it will be too loud to hear.
I was about to witness the heart of the supercomputing center: 27 large containers, in neat rows, each housing multiple processors with speeds and abilities too great for my mind to wrap around. Inside, the temperature is by turns arctic and tropic, so-called cold rows alternating with hot fans operate around the clock to cool the processors as they churn through millions of giga, mega, tera, peta and other ever-increasing scales of data bytes. In the cool rows, robotic-looking lights blink green and blue in orderly progression. In the hot rows, a jumble of multicolored wires crisscrosses in tangled skeins.
In the corners stood machines that had outlived their heyday. There was Sherlock, an old Cray model, that warmed my heart. There was a sad nameless computer, whose anonymity was partially compensated for by the Warhol soup cans adorning its cage (an homage to Warhols Pittsburghian origins).
And where does Libratus live, I asked? Which of these computers is Bridges, the computer that runs the AI Sandholm and I had been discussing?
Bridges, it turned out, isnt a single computer. Its a system with processing power beyond comprehension. It takes over two and a half petabytes to run Libratus. A single petabyte is a million gigabytes: You could watch over 13 years of HD video, store 10 billion photos, catalog the contents of the entire Library of Congress word for word. Thats a whole lot of computing power. And thats only to succeed at heads-up poker, in limited circumstances.
Yet despite the breathtaking computing power at its disposal, Libratus is still severely limited. Yes, it beat its opponents where Claudico failed. But the poker professionals werent allowed to use many of the tools of their trade, including the opponent analysis software that they depend on in actual online games. And humans tire. Libratus can churn for a two-week marathon, where the human mind falters.
But theres still much it cant do: play more opponents, play live, or win every time. Theres more humanity in poker than Libratus has yet conquered. Theres this belief that its all about statistics and correlations. And we actually dont believe that, Nystrom explained as we left Bridges behind. Once in a while correlations are good, but in general, they can also be really misleading.
Two years later, the Sandholm lab will produce Pluribus. Pluribus will be able to play against five players and will run on a single computer. Much of the human edge will have evaporated in a short, very short time. The algorithms have improved, as have the computers. AI, it seems, has gained by leaps and bounds.
So does that mean that, ultimately, the algorithmic can indeed beat out the human, that computation can untangle the web of human interaction by discerning the little tactics of deception, of asking yourself what is the other man going to think I mean to do, as von Neumann put it?
Long before Id spoken to Sandholm, Id met Kevin Slavin, a polymath of sorts whose past careers have including founding a game design company and an interactive art space and launching the Playful Systems group at MITs Media Lab. Slavin has a decidedly different view from the creators of Pluribus. On the one hand, [von Neumann] was a genius, Kevin Slavin reflects. But the presumptuousness of it.
Slavin is firmly on the side of the gambler, who recognizes uncertainty for what it is and thus is able to take calculated risks when necessary, all the while tampering confidence at the outcome. The most you can do is put yourself in the path of luck but to think you can guess with certainty the actual outcome is a presumptuousness the true poker player foregoes. For Slavin, the wonder of computers is That they can generate this fabulous, complex randomness. His opinion of the algorithmic assaults on chance? This is their moment, he said. But its the exact opposite of whats really beautiful about a computer, which is that it can do something thats actually unpredictable. That, to me, is the magic.
Will they actually succeed in making the unpredictable predictable, though? Thats what I want to know. Because everything Ive seen tells me that absolute success is impossible. The deck is not rigged.
Its an unbelievable amount of work to get there. What do you get at the end? Lets say theyre successful. Then we live in a world where theres no God, agency, or luck, Slavin responded.
I dont want to live there, he added I just dont want to live there.
Luckily, it seems that for now, he wont have to. There are more things in life than are yet written in the algorithms. We have no reliable lie detection software whether in the face, the skin, or the brain. In a recent test of bluffing in poker, computer face recognition failed miserably. We can get at discomfort, but we cant get at the reasons for that discomfort: lying, fatigue, stress they all look much the same. And humans, of course, can also mimic stress where none exists, complicating the picture even further.
Pluribus may turn out to be powerful, but von Neumanns challenge still stands: The true nature of games, the most human of the human, remains to be conquered.
Maria Konnikova is the author, most recently, of The Biggest Bluff. She is a regularly contributing writer for The New Yorker, the author of two previous New York Times best-sellers, and a professional poker player.
Continue reading here:
The Deck Is Not Rigged: Poker and the Limits of AI - Undark Magazine
- Most Poker Rooms Remain Closed As Gambling Markets And Casinos Reopen In The United States - CardPlayer.com [Last Updated On: June 17th, 2020] [Originally Added On: June 17th, 2020]
- New poker rules in the light of the COVID-19 pandemic - Tampa Bay Times [Last Updated On: June 17th, 2020] [Originally Added On: June 17th, 2020]
- Genting Ceases All Live Poker Activity in the United Kingdom - PokerNews.com [Last Updated On: June 17th, 2020] [Originally Added On: June 17th, 2020]
- Ike Haxton Raises Concerns About the Health of Live Poker Players - HighstakesDB [Last Updated On: June 17th, 2020] [Originally Added On: June 17th, 2020]
- King's Casino reopening poker room soon with plenty on the schedule - World Casino Directory [Last Updated On: June 17th, 2020] [Originally Added On: June 17th, 2020]
- Advt: 5 reasons why everyone is hooked to online poker! Have you joined the party yet? - Times of India [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Orange City OKs more poker players, and Greyhound racing in Daytona is over - The West Volusia Beacon [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- How Coronavirus changed the perception of Poker - CalvinAyre.com [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Natural8 2020 WSOP Online Hand of the Week: A Wholesome Hand for Ryan Depaulo - PokerNews.com [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- How Has Poker Changed Over Many Years? - Loop21 [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Trump's approach to politics bears the hallmarks of a bad poker player, author says - CBC.ca [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Today's lottery news: the Poker Lotto numbers and results for Monday July 20, 2020 are here... - Born2Invest [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Key Points To Consider When Choosing Online Poker Tables - Top10PokerWebsites [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- NJ Special Legislation and the Impact on Online Casino Players in NY - Blog - The Island Now [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- The Phil Ivey Of Poker - HBCU Buzz [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Venetian poker room going to 8-handed tables - Las Vegas Review-Journal [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- Chasing Poker Greatness: #68 Andrew Brokos: Co-Host ThinkingPoker Podcast & Author of Play Optimal Poker - Pokerfuse [Last Updated On: July 21st, 2020] [Originally Added On: July 21st, 2020]
- FULL-HOUSE VS QUADS in the WSOP Main Event! Could Selbst have gotten away? - PokerNews.com [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- Error in partypoker US Online Network PKO Series Finale Results in Nearly $50K Overlay - PokerNews.com [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- Facebook develops AI algorithm that learns to play poker on the fly - VentureBeat [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- Why Negreanu thinks Ivey is the best poker player in the world - Comcast SportsNet Chicago [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- An Open Letter To The WSOP: Why Mixed Games Are Good For Poker Player Development - Poker News - CardPlayer.com [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- What Site Offers the Best Online Poker Bonus in 2020? - PokerNews.com [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- Man, 60, in custody in connection to death of professional poker player Susie Zhao - Detroit Free Press [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- 60-year-old man arrested in connection with Oakland Co. professional poker player's death - WXYZ [Last Updated On: August 3rd, 2020] [Originally Added On: August 3rd, 2020]
- VIEW: Poker has a lot to offer to the Robinhood generation - CNBCTV18 [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Key Elements to Consider When Searching for the Best Poker Game Software Development Company - Communal News [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- LOCK, STOCK AND TWO SMOKING BARRELS To A BIG HAND FOR THE LITTLE LADY : Best Poker Movies To Perfect Your Game Plan - The Fan Carpet [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Casino Games: Online Poker | What Happened To Full Tilt Poker? - GamingTodaySlotsToday [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Who Are the All-Time Leaders in Poker in North America? - Poker News Daily [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- UPDATE: One Arrested After Poker Game Argument Leads to Machete Attack in Lexington Park | Southern Maryland News Net - Southern Maryland News Net [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Caribbean Sports - Best Caribbean Poker Tips & Tricks To Win - Caribbean and Latin America Daily News - News Americas [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Author and former professional poker player Annie Duke on how conspiracy theories gain ground - TechCrunch [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Poker Market Size and Growth By Leading Vendors, By Types and Application, By End Users and Forecast to 2020-2027 - Scientect [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- When Will Live Poker Rooms Reopen in Pennsylvania? - Play Pennsylvania [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- 5 of the best gambling apps to play real money poker games - European Gaming Industry News [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Poker - Wikipedia [Last Updated On: August 26th, 2020] [Originally Added On: August 26th, 2020]
- Andre Akkari Brings eSports and Poker Together as Industry Revels in Good News - HighstakesDB [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- Five True-Life Poker Stories Primed To Be Hollywood Blockbusters - PocketFives [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- Christopher Nunnally Wins Pearl River Poker Open $600 Main Event ($57835) - PokerNews.com [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- What Losing 40 Tournaments In A Row Should Tell You About Your Poker Game - Poker News - CardPlayer.com [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- Phil Ivey's Most Memorable Hand that Changed the Course of Poker History - HighstakesDB [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- Legendary poker player goes all in on Bitcoin, liquidates Ether and Monero - Crypto News Flash [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- Poker Club: All Bluff And Bluster | Screen Rant - Screen Rant [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- What Does Another WSOP Main Event Mean for Poker? - PokerNews.com [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- 2020 PokerNews Holiday Gift #4: SimplePoker.com GTO Poker Solvers and Tools - PokerNews.com [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- Why Casino Royale Director Sweated the Poker Scenes Over Any James Bond Action - MovieWeb [Last Updated On: November 29th, 2020] [Originally Added On: November 29th, 2020]
- What Kinds Of Basics Beginners Should Know In Live Poker Gambling? - - VENTS Magazine [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Poker Legend Maria Ho Details Her Experience on 'The Amazing Race' and 'American Idol' (Exclusive) - PopCulture.com [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Great minds: 10 players who excel at both chess and poker - chess24 [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Head of Marketing- Poker Powher - PEAK6 - Built In Chicago [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- 'Twas The Night Before Christmas - Poker Edition - PocketFives [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Maria Ho Reflects on Hall of Fame Poker Career, Teases Her Future Plans (Exclusive) - PopCulture.com [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Bitcoin And Crypto Trading Tips From Poker World Champion Annie Duke - Forbes [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Chemistry Lessons: Building the Perfect High Stakes Poker Cast - PocketFives [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Poker Stories of the year: Negreanu and Polk's high stakes feud - CalvinAyre.com [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- 2021 Poker Predictions: Will COVID-19 Force a WSOP Cancellation? - CardsChat.com [Last Updated On: December 26th, 2020] [Originally Added On: December 26th, 2020]
- Huck Seed inducted to Poker Hall of Fame - CalvinAyre.com [Last Updated On: January 7th, 2021] [Originally Added On: January 7th, 2021]
- Conor Beresford Wins 2020 Online Poker Player of the Year - PocketFives [Last Updated On: January 7th, 2021] [Originally Added On: January 7th, 2021]
- WPT Readies for Live Poker in Las Vegas and Florida - LegalUSPokerSites.com [Last Updated On: January 7th, 2021] [Originally Added On: January 7th, 2021]
- Run It Once Poker in 2020: Innovation, Promotions and Growth - Pokerfuse [Last Updated On: January 7th, 2021] [Originally Added On: January 7th, 2021]
- Jail for woman who attacked neighbour with poker in Cork town - EchoLive.ie [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- Poker Coming Back to Detroit This Week - USGamblingSites.com [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- This woman used her Psychology Ph.D. to become a poker star - Yahoo News [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- Looking Ahead To The World Series Of Poker Finale - LA Canyon News [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- Sheldon Adelson Died and the Poker Community has Mixed Feelings - CardsChat.com [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- Poker Is on the Cards for Detroit This Friday - GamblingNews.com [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- FIVE THINGS: The Poker Hall of Fame Deserves a Next Level Upgrade - PocketFives [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- Tony G Resigns from Parliament; Will He Head Back to Poker? - PokerNews.com [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- High Stakes Poker S8, E5: You Won't Believe the Pot Tom Dwan Won - PokerNews.com [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- 5 key concepts in poker and chess | chess24.com - chess24 [Last Updated On: January 17th, 2021] [Originally Added On: January 17th, 2021]
- Several States Begin To Lift Casino Closures And Ease Restrictions - CardPlayer.com [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- Jack Hardcastle Wins the WPT Montreal Main Event at partypoker ($447859) - PokerNews.com [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- Los Angeles Poker Rooms Given Green Light to Reopen this Week - CardsChat.com [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- Partypoker US Network Posts Big Online Poker Revenue Gains in New Jersey During the Final Quarter of 2020 - Pokerfuse [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- 888 Executive: US Online Poker to Benefit from Latest Wire Act Ruling - Casino.Org News [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- The FIVES Poker Podcast: WPT in the Headlines, Polk Tilts Negreanu - PocketFives [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- Justice Kagan: Poker Shark - Above the Law [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- Is online poker in the cards for North Dakota? - The Dickinson Press [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]
- Online poker in the cards for North Dakota? - INFORUM [Last Updated On: January 29th, 2021] [Originally Added On: January 29th, 2021]