Microsoft just revealed a new tool that’s capable of analyzing photos to identify human emotions.
For example, imagine sitting in a movie theater with a camera pointed at the audience. Microsoft Research program manager Ryan Galgon said a camera with this technology could tell film producers at any particular moment how many people in the audience were surprised, sad or just didn’t react at all.
Store owners could do the same thing to see how customers are reacting to product displays, and app makers could sift through entire photo libraries and save just the happiest images.
These are some of the possible scenarios, but — really — Microsoft has no idea how developers will use the technology.
It’s all part of the company’s recently launched Project Oxford. Microsoft is calling on its years of experience in machine learning and artificial intelligence to build sophisticated tools that it makes available for anyone for free.
“Not every developer is going to become a machine learning expert,” Galgon said. “Project Oxford is all about bringing those tools to every developer… to see what people can build.”
Developers can build the emotion-detecting tool into their apps. The software sends Microsoft still images and receives back a numeric score that rates the degree to which each face shows eight different emotions: anger, contempt, fear, disgust, happiness, neutral, sadness or surprise.
Of course, the platform can also be tweaked for more entertaining uses. Microsoft, for instance, recently launched mymoustache.net, which used the technology to rate user submitted photos based on the quality of their mustaches.
Microsoft has posted a live demo of its emotion sensing tool online, and it worked pretty well when I had a chance to play with it. The app doesn’t read minds or pick up on emotions that are invisible to the human eye. Instead, it’s more about automating what a human brain does in a split second when presented with a photo.
This is just one API to come out of Project Oxford. The program debuted in May with an original set of tools that can do things like spot racy photos, understand natural human speech and identify users with facial recognition software. Microsoft rolled out its next batch of tools, including the emotion detector, on Wednesday.
Here are some of the other tools the company announced today:
Spell check: Microsoft’s spell check tool, which became available on Wednesday, is based on machine learning, so it’s always getting smarter and learning new slang, brand names and common grammatical errors.
Video: Microsoft is making many of the same technology used in its Hyperlapse video app available to the public by the end of the year. Tools that do things like detect faces, track motion and image stabilization will be part of the API.
Voice recognition: This tool, which will be out by the end of the year, learns a user’s voice and then recognizes whether or not that’s who’s speaking. Microsoft says it doesn’t recommend app makers use this to replace traditional passwords, but it could be used as an added layer of security.
Custom Recognition Intelligent Services: This tool, known as CRIS, is only available to hand-selected beta testers for now, but will be made more broadly available sometime in the future. It’s designed to enable speech recognition in challenging environments, such as a noisy shopping center or with a child who doesn’t enunciate.
