Microsoft’s research arm is working on a portable mat that can sense objects and then project them on to a smart device or computer screen, with a goal of closing the gap between the physical and digital worlds.
Project Zanzibar uses the concept of near field communication, a wide-ranging technology that lets two nearby devices interact in situations like using a smartphone to pay at the register, to digitize real-world items through low-cost stickers. Microsoft researchers from the U.K. and company HQ in Redmond, Wash. teamed up on the project, which will be presented at the 2018 Conference on Human Factors in Computing Systems in Montreal later this month.
“We began with a simple thought — what if we could blur the divide between the physical and digital worlds? What if you could play with physical toys, cards, and blocks, and watch your actions come alive on the screen,” the team states in an introduction to the project on the Microsoft Research blog.
The team detailed two main uses as it continues to build the technology: play and learning. Project Zanzibar aims to give kids the opportunities to beam their favorite toys into the digital realm and then make movies, photos or other art showcasing the items. Card games could be aided by accompanying action on screen. The researchers worked with lego-like toys in one example, creating interactions between them on screen when placed next to each other in the real world.
“The sensing ability of the mat can be extended by the toy pieces, for example when the pirate climbs the tower, the mat detects when he reaches the top,” according to the blog post. “Toy pieces can also become inputs; when the pirate fires his cannon, players hear the boom and later a splash when the cannonball lands in the sea.”
Inspired by the Montessori Method of early childhood education, the team showed a program with letter plates that can be traced by hand to work on writing out words. Eventually, the project aims to produce a variety of simple spelling and coding apps.
