The Ms. in “Ms. Pac-Man” might as well stand for Microsoft now. The software giant has achieved the highest score possible on the vintage video game by using a type of artificial intelligence called reinforcement learning — and the achievement is about more than just bragging rights at the arcade.
The maximum score of 999,990 — on the Atari 2600 version of the game — was registered by a team from Maluuba, a Canadian deep-learning startup acquired by Microsoft earlier this year, according to a post on Microsoft’s Next blog. The score is four times higher than the best score by a human.
To get the high score, the team divided the task of mastering Ms. Pac-Man into separate parts, which were distributed among AI agents, Microsoft reported. Called Hybrid Reward Architecture, Maluuba’s method used more than 150 agents, each working with others to conquer the game, with some being rewarded for tasks such as eating pellets or chasing down fruit.
A top agent then used input from all the other agents to decide where to move Ms. Pac-Man to stay alive and rack up points.
Microsoft reported that reinforcement learning is the counterpart to supervised learning, “a more commonly used method of artificial intelligence in which systems get better at doing something as they are fed more examples of good behavior.” And experts believe that it could lead to AI agents that “can make more decisions on their own, allowing them to do more complex work and freeing up people for even more high-value work.”
Steve Golson, a co-creator of the arcade version of “Ms. Pac-Man,” told Microsoft that it makes him smile to learn that his game is widely used as a testing ground for AI research. He said it makes sense, in that the game was purposely designed to be more difficult and have less predictability than regular “Pac-Man.”
For humans, this made the game harder to beat. And it caused them to spend more money trying.
Read Microsoft’s full blog post here, and the Maluuba research paper on the project here.
