A new virtual reality system developed by Microsoft researchers lets users walk through the real world while fully immersed in a virtual one — using a mobile VR headset and an array of sensors to detect obstacles and adapt the digital scene to the surrounding environment as they go.
Called “DreamWalker,” the Microsoft Research prototype is one of the projects to be presented by the company in New Orleans this week at the ACM Symposium on User Interface Software and Technology (UIST) 2019. It’s the latest example of the company’s longstanding work in virtual and augmented reality applications.
“Provided with a real-world destination, DreamWalker finds a similar path in a pre-authored VR environment and guides the user while real walking the virtual world,” the researchers explain in their paper.
“To keep the user from colliding with objects and people in the real-world, DreamWalker’s tracking system fuses GPS locations, inside-out tracking, and RGBD frames to 1) continuously and accurately position the user in the real world, 2) sense walkable paths and obstacles in real time, and 3) represent paths through a dynamically changing scene in VR to redirect the user towards the chosen destination.”
They write that they demonstrated the systems versatility by “enabling users to walk three paths across the large Microsoft campus while enjoying pre-authored VR worlds, supplemented with a variety of obstacle avoidance and redirection techniques.”
In their testing, they explain, “8 participants walked across campus along a 15-minute route, experiencing a virtual Manhattan that was full of animated cars, people, and other objects.”
Researchers on the project are Stanford University PhD student Jackie Yang, who worked on the project as a Microsoft Research intern; along with Microsoft researchers Eyal Ofek, Andy Wilson, and Christian Holz, now an ETH Zürich professor. Here’s their full paper.
Related projects to be shown by Microsoft at the conference include a new eye-tracking technology that changes elements of a scene when the user isn’t looking, and a new haptic feedback controller that “uniquely emulates the human sense of touch,” as explained in this Microsoft post about the projects.
While these aren’t full-fledged products, Microsoft research can inform and help to inspire the direction of the broader company. Microsoft is involved in augmented reality for a variety of commercial applications through its HoloLens headset, and Windows Mixed Reality headsets created by the company’s hardware partners.
