In a bid to improve on the native iPhone camera app, Microsoft Research today released a new smart camera app for iPhone, Microsoft Pix, that specializes in photos of people and uses artificial intelligence to remove common smartphone camera issues like blurry or backlit shots.
As Microsoft pulls back from its own phone hardware business, it has focused more on acquiring and releasing apps for Android and iOS. It has beefed up some of its signature offerings like Outlook and Bing to include better integration with other apps.
As soon as a user opens the app, Pix is thinking, adjusting settings automatically to compose the best possible shot. That means users don’t have to toggle through various modes. The app can recognize faces and see when eyes are open or closed.
For every photo, Pix shoots 10-frame bursts that start before the user pushes the button. It then goes through the burst and picks out the best photo or couple of photos. Pix looks at all images in a burst and uses information from each one — including the ones it will discard — to enhance the best shots.
“While the app is selecting and enhancing the best of the burst, another set of algorithms sorts through the frames to determine whether any motion would make for an interesting looping video within the image, such as a person’s hair tussled by the wind or the cascade of a waterfall in the background. If so, the app will loop the specific motion for a Harry Potter-esque effect called Live Image,” explains a Microsoft blog post about Microsoft Pix.
This is possible because of algorithms that stabilize photos and eliminate the effects of shaky hands.
The app syncs with iPhone’s camera roll, and it works with iPhones dating back to the 5S model as well as newer iPads. Pix also includes Hyperlapse for mobile, which uses algorithms to stabilize videos and create time lapses.
Pix is the latest example of a shift within Microsoft Research to focus more on getting projects out of the labs and into the market. Previously, the division focused more on basic research, but that is shifting under CEO Satya Nadella. The risk with the new approach is that researchers could become too focused on money-making projects instead of ambitious ideas that could pay off years down the road. But Microsoft’s new model is more in line with the approach taken by Facebook and Google, and it has helped lead to the creation of new futuristic products like Skype Translator and the HoloLens.
