PITTSBURGH — In preparing for this story, I was all set to describe in detail how I had been soundly defeated by Baby Tartanian8, a poker bot built by some of smartest folks I’ve ever come across from Carnegie Mellon University. And then a funny thing happened, I actually came out ahead!
The victory comes with caveats aplenty — we played a minuscule sample size of 20 hands, I only finished up 103 chips in a heads-up format with blinds of 50 and 100, and I folded on the final hand in cowardly fashion — but it still counts. For anyone looking for a safe investment in today’s volatile markets, the best thing to do would be to fund my entry into the World Series of Poker Main Event this year. You won’t be disappointed.
I’m sure you’re all wondering, and I was too: why a poker bot? Poker is what the CMU researchers call an imperfect information game. That means there is more than one player and unknown variables. Baby Tartanian8 runs off a server back at CMU and uses game theory and Nash Equilibrium, a concept concerning optimal decision making in adversarial situations.
The researchers didn’t teach the bot how to play. It is able to read the rules, and then an algorithm runs on a supercomputer for a few weeks developing a strategy that is stored on a server and referred to during games.
“It tries to solve the game, so it takes a model of the rules of the games — at this point I can do these things, and the opponent can do these things, and then I can do those things — and then it figures out how to put probabilities on all the different moves for both players, but that’s a very hard thing to do,” explained Tuomas Sandholm, a professor at CMU’s School of Computer Science who co-created Baby Tartanian8 with PhD student Noam Brown.
It was a fascinating experience to jump into the game. Sandholm had just wrapped up speaking on a panel at the headquarters of language learning startup Duolingo. There we sat in the Duolingo cafeteria, talking about the nuances of the game while 250 techies and community leaders networked and munched on pirogies and downed Big Hop beers made by Pittsburgh’s East End Brewing Co.
I consider myself a solid poker player. I frequently walk away victorious in my irregular home game and usually come out ahead at the tables in Las Vegas. For a period in the mid 2000s — the heyday for Texas Hold ‘Em — I even managed to pay some school bills with online poker winnings. In getting ready for this assignment though I got deep in my own head, assuming Tartanian would wipe the floor with me, no matter the number of hands we played. Before the meeting I had a placeholder headline at the top of this post that started “Carnegie Mellon’s champion poker bot kicked my ass …”
Had we played even a few more hands I probably would have went down quickly as the bot was definitely a better player than me.
Now for the game. I’m about to jump into some wonky poker strategy here, so if you’re not familiar with the basics of Texas Hold ‘Em, the most popular version of the game right now, check out this introduction.
I started off hot, winning four of the first five hands to build a 4,000-chip advantage over my opponent and revive my confidence. I twice nailed three-of-a-kind on the final of the five communal cards — also known as the river — and picked up a lot of chips that way. I ended up giving most of it back, sometimes because I just wanted to see Tartanian’s cards and understand its strategy, and in others instances I just got outsmarted.
It was clear from our session that Sandholm knows the game extremely well, no surprise given all the time he’s spent building these programs. On a couple occasions he second-guessed my moves, and each time I listened to him it worked, including one situation in which he called out the bot’s hand exactly and urged me to make small bets to extract maximum value from my opponent. Thanks for the tips Tuomas!
Tartanian was a cagey, and often frustrating, opponent. In Texas Hold ‘Em, few hands make it all the way to the end with both players showing their cards. So in 20 hands it was hard to pick up any meaningful trends. The bot kept bet sizes consistent throughout, keeping me guessing as to what Tartanian was up to. The bot also acted instantaneously, rather than stopping to think about a move for a second, which threw me off a little.
However, I did manage to learn a few bits of information on the playing style of Baby Tartanian8. If I showed weakness, choosing to not bet in some occasions or just call bets early in the hand rather than raise, it would often throw out very small bets. It was almost begging me to call its bets, but the bot did a good job of making this move in a variety of situations and frustrating the hell out of me in the process. On several occasions it slow-played top hands and in others it was over-aggressive, exactly the mix of strategies you would expect to encounter going up against a top-shelf poker player.
On a hand where I started with Ace-King of the same suit, the best two cards you can get outside of a pair, Tartanian tripped me up with a huge raise. That put pressure on me to commit a major chunk of chips to a great starting hand that hadn’t connected with any cards on the board. Over Sandholm’s advice, I folded.
I made some moves I might not otherwise make — and that wouldn’t be the best plays in a long-term setting — just to see how the bot would react. A couple of times I was able to bluff it out of hands with bad starting cards like 7-5.
To get the full portrait of the bot’s skills — and to be thoroughly crushed — I’d need to play a lot more hands, Sandholm tells me.
“To know who is actually better you have to play 10s of thousands of hands, so at 20 hands you have probably a 50 percent chance of ending up ahead,” Sandholm said.
Baby Tartanian8 is one of several poker bots built at CMU. It has won competitions against other poker AI programs, but it does not regularly battle humans. CMU’s top poker bot, Libratus, bested some of the best poker players in the world here in Pittsburgh last year. A big difference between Libratus and other poker bots is its ability to improve and learn over time. It is able to recognize and patch holes in its game, making it an even more formidable player as the game progresses.
Sandholm and his team have been working on this technology for more than 15 years and actually licensed it to Strategic Machine Inc., a company founded by Sandholm to apply strategic reasoning technologies to a range of applications that extends well beyond games. Any situation where there is more than one player involved and a series of unknown variables makes sense as an outlet. The range includes everything from military planning and strategy, dynamic pricing changes in retail settings, auctions and bidding for prizes like movie and streaming rights and much more.
And since this is Pittsburgh, a hub for self-driving vehicle research, there are applications to autonomous cars as well.
“You have to follow the rules of the road, but that leaves a lot unspecified,” Sandholm said. “How do you merge, for example? Do you merge like people do where they slow down and look at each other, or would it be better if those situations between those two fleets are negotiated in advance so they can just merge at full speed.”
