The best poker players in the world can cash in on millions of dollars in a game. Played in casinos, poker clubs, private homes and on the internet, the game demands skill and strategy.
Now scientists have created an artificial intelligence (AI) bot that can best even the top human players. And this new AI won at six-player poker. Bots were already dominant at two, or three-player poker, but six players is much harder. The feat represents a major breakthrough in artificial intelligence that could one day apply to far beyond card games to everything from cybersecurity to navigating self-driving cars.
“This research isn’t really about poker,” said computer scientist Noam Brown, who authored the work while completing his doctoral degree at Carnegie Mellon University and working as a research scientist for Facebook AI.
“It’s about developing AI that can handle hidden information in a complex multi-participant environment.”
Touchstone Task
In any game of poker, the goal is to win the “pot,” the collection of bets players make throughout each deal. Players win by having the highest-ranking set of five cards in hand or by making a bet that no other player matches. Because there are multiple players, participants must work with imperfect information about their opponents, a situation that’s previously made it difficult for AI to succeed.
“Poker is a useful benchmark for progress on this more general problem because in poker we can objectively measure performance against professionals who have dedicated their lives toward reaching the peak of human performance in this game,” Brown explained.
Two years ago, Brown and a team of researchers developed another AI called Libratus that beat poker pros playing heads-up no-limit Texas hold’em, a two-player version of the game. But since most real-world AI applications involve more than two participants, developing a bot that could win in six-player no-limit Texas hold’em poker – the most popular version of the game – was a long-standing challenge.
Now the researchers have revealed their improved AI, which they call Pluribus. Pluribus first played against copies of itself to create what the researchers dub a “blueprint strategy.” As the AI plays, it figures out what actions lead to better outcomes. Then, when playing against human opponents, Pluribus improves the blueprint strategy by searching in real time for a strategy that better suits the circumstances of the current game.
Superhuman Strategy
The overall strategy led Pluribus to beat some of the best players of the game for the first time, the researchers announce Thursday in the journal Science. The AI had a very high win rate when it competed against five professional poker players in 10,000 hands of the game over 12 days. Pluribus won at a rate of 48 milli big blinds per game, which is a measure of money won based on how much the second player put in the pot. Forty-eight is considered a very high win rate.
In another round where one human elite played 5,000 hands of poker against five copies of the Pluribus, the AI beat the human by 32 milli big blinds per game. For comparison, poker superstar Chris “Jesus” Ferguson, who has won nearly 10 million dollars in live earnings, lagged behind Pluribus by 25 milli big blinds per game.
“Pluribus plays at a superhuman level, and defeats elite human professionals in six-player poker even when they have time to observe the bot’s strategy and adapt to it,” Brown said.
“In the future I can see this research being applied to everything from cybersecurity to combating fraud to navigating traffic with a self-driving car,” he added.
