For decades, the games that put artificial intelligence to the test have been played human vs. machine – whether it’s checkers, chess, Go, poker, StarCraft or “Jeopardy.” Why isn’t there a game where the AI and the human are on the same side?
Now there is, and you can play, too.
Researchers at Seattle’s Allen Institute for Artificial Intelligence are taking the wraps off Iconary, a Pictionary-type puzzle game in which an AI and human players take turns putting together pictures and guessing what phrases the pictures signify. Anyone can play the game with the AI agent, nicknamed AllenAI, by going to Iconary.AllenAI.org.
But it’s not just a game.
Iconary is also a platform designed to see how far the institute, known as AI2, can go in its quest to perfect what the game’s inventors call “common-sense AI” — that is, the ability to understand everyday concepts and make abstractions to deal with new situations.
“We wanted to study the kinds of games that teach us more about the underlying assumptions and knowledge about how the world works,” said Ali Farhadi, a University of Washington computer scientist who heads up AI2’s Perceptual Reasoning and Interaction Research team, or PRIOR.
Farhadi said the structure of the game meshes with how he and his colleagues believe AI will actually be used in the real world, in contrast to the Terminator-tinged dystopia that comes up so often in the debate over the future of AI.
“Thinking about this space, and also all the noise that’s out there saying that AI’s going to wake up tomorrow and rule the world, and we’re all going to be slaves to the AI and it’s going to kill everybody … we don’t believe that,” he said. “The future that we believe in is an AI who can help human beings, and actually collaborate with human beings in achieving a goal.”
Testing for theory of mind
A key concept for understanding collaboration, and human-style intelligence in general, is a little something called “theory of mind” — basically, our ability to put ourselves in someone else’s shoes and see reality through their eyes.
Theory of mind is what’s thought to distinguish human cognition from animal cognition, although some scientists say they see signs of the trait in chimpanzees, birds and other creatures. It’s what makes it possible to imagine alternate scenarios and formulate strategies for those scenarios, even if they’ve never been encountered before.
Farhadi said the theory-of-mind approach is different from the typical machine-learning approach to AI, which involves training the software on myriad real-world examples that have been encountered before. Iconary could provide a way to get at the distinction.
Iconary grew out of the PRIOR team’s search for alternatives to the adversarial games so often used for AI research.
“We stumbled upon Pictionary as a perfect testbed for us,” Farhadi said. “If you are playing a Pictionary game successfully, you are collaborating with your teammate. You are connecting to your teammate at a deep level and getting into his or her thought. You need to understand his or her way of thinking about the problem … and you need the predict the future behavior of your teammate, based on the kind of feedback you provide.”
Pictionary checked all the boxes that Farhadi and his fellow researchers wanted to fill, but they had to change the game to have it work on a computer.
The PRIOR team created a vocabulary of 1,200 icons, ranging from stick figures and arrows to pictures of fruit and buildings. When AllenAI is doing the drawing, it will formulate a phrase – for example, “drinking fruit punch” – and then select the icons for a mouth, a glass and an orange to represent the phrase.
If you’re not able to guess the correct phrase, AllenAI will try again, focusing on the incorrectly guessed words. For example, if you guess “drinking orange juice,” AllenAI will fill the screen with fruit icons to steer you in the right direction.
“Growing up, I played a lot of Pictionary … I think AllenAI is a better drawer than my family,” AI2 principal software engineer Sam Skjonsberg joked.
When it’s your turn, you can draw rough sketches of your clues and pick out the icons that come closest to your clues from a menu. Then AllenAI will try guessing the phrase. If it can’t figure out the puzzle, it’ll ask for more clues.
For the researchers, the most interesting part has to do with how AllenAI puts together the clues and figures them out.
“AllenAI has never before encountered the unique phrases in Iconary, yet our preliminary games have show that our AI system is able to both successfully depict and understand phrases with a human partner, with an often surprising deftness and nuance,” AI2 CEO Oren Etzioni, Ani Kembhavi and Carissa Schoenick say in a Medium blog post.
Kembhavi, a research scientist at AI2, said AllenAI is catching on to our human ways.
“It has started understanding the meanings of and the usages of abstraction,” he said. “For example, it will put an arrow when it wants to show, let’s say, a state change. If the phrase is ‘ice melting,’ it will show ice with an arrow and water — which is a state change.”
Unleashing AllenAI
Now it’s time to send AllenAI out of the lab and into the wider world. “We need large-scale feedback from human players about the game to tell us what’s working and what’s not working, so we can actually devise the next steps,” Farhadi said.
Iconary might be tweaked to provide AllenAI with icons for even more abstract concepts such as “roommate” or “governor.” And because Iconary is open-source, like all the data generated by the late billionaire philanthropist Paul Allen’s scientific ventures, other AI researchers can tweak it for non-commercial purposes. (For what it’s worth, the makers of Pictionary haven’t raised legal objections to Iconary.)
Eventually, AI2’s team might mix things up so that you’re playing with AllenAI only part of the time, and with human players the rest of the time.
“At the end I’m going to ask you if you could distinguish the fact that you played a human or you played a machine,” Farhadi said. “We’re going to basically see how much data we can get out of this as a first step, and that will give us a benchmark to evaluate how far we are from a Turing-qualified intelligent machine.”
Farhadi suspects that Iconary will have even more staying power than chess, Go or StarCraft as a tool for exploring the frontiers of artificial intelligence.
“This is a whole different way of thinking about AI and games, and what you can actually learn from playing a game other than just being the best at playing that game,” he said. “Because of that, I believe we are going to spend a fair amount of time on this. I certainly hope that looking back at this in five years, this will really be … an icon in that space.”
