mspatent
A diagram from a newly released Microsoft patent application, “Synchronizing virtual actor’s performances to a speaker’s voice.”

A new Microsoft patent shows how a head-mounted display could transform the experience of reading books — synchronizing a virtual character’s movements and lips to match the words being read aloud from the text.

The patent, awarded to the company today, was initially filed three years ago, but it provides a sense for the direction the company could be headed. One of the inventors listed on the patent is Alex Kipman, who is leading development efforts for HoloLens, the Redmond company’s upcoming “blended reality” headset.

Win10_HoloLens_Hardware_Web

The patent describes the use of augmented reality tags in books, detected by the headset to trigger the display of specific characters or scenes in the environment around the person reading the book. Books could come with these tags or be retrofitted with them.

Sensors in the headset would detect the pace at which the book is being read, and synchronize the movement of the character’s lips to the words in the text.

According to the filing, the technology would work with a variety of “reading objects,” which could include “a book, magazine, journal, newspaper, or work of literature fixed in a tangible medium of expression,” which would presumably include e-books.

As noted in the patent, the approach could be particularly useful when kids are learning to read, or having books read to them by parents, to help them engage with the content.

Microsoft unveiled the Windows 10-powered HoloLens in January, without announcing pricing or a specific release date. Most recently, the company has been working to get developers making new apps and experiences for the device.

Editor’s Note: We discovered this filing through an alert from Sqoop, a Seattle-area startup developing a public-records search and notification service for journalists. 

Like what you're reading? Subscribe to GeekWire's free newsletters to catch every headline

Job Listings on GeekWork

Find more jobs on GeekWork. Employers, post a job here.