The alarm sounds. A wall of heavy machines begins to rattle, as something has clearly gone wrong. Playing the role of a maintenance engineer at a high tech manufacturing facility, I have no clue what to do. So I call my boss “Michael” to walk me through it.
But this is not your average call. I have a Microsoft HoloLens mixed reality headset strapped on and I’m using a new app called Remote Assist, one of two new “mixed reality” tools announced at Microsoft Build this week that I got to try out, to figure out how to fix this real machine.
The video call happens on Microsoft Teams and lets Michael — who is a real person in a nearby office somewhere, not a simulation — see everything I’m looking at through HoloLens. Using the signature gesture of the HoloLens — taking my index finger and pinching it down to execute the “air tap” — I pin my video call with Michael to a wall and turn my attention to a real-life control panel. He directs me to turn off the machinery, dropping in arrows to show the correct procedure to do so.
Next, I move over to a separate machine and find that a belt has fallen off its pulleys. Michael draws me a roadmap in my field of vision to show how to reattach the belt. But still, something is wrong. I move over to another panel and get a bit lost, so I pull out a digital version of the machine’s manual and pin it to a nearby wall.
I find a faulty wire and replace it. Then it’s back to the control panel where I start everything back up with my fingers crossed. And success, nothing blew up on me! I am now qualified to fix and maintain large pieces of factory machinery.
Remote Assist, and the other major new app announced this week called Layout, are part Microsoft’s latest HoloLens play to appeal to what it calls “first-line workers.” Those are the people who don’t sit at desks all day and have direct interactions with either customers or equipment.
Microsoft wants to sell HoloLens to a larger variety of customers, including manufacturers, healthcare companies, design and construction firms, not just for planning projects but for workers in factories or the field. These industries, Microsoft claims, have been left behind by the advances in technology that have made it easier for knowledge workers and creative types to do their jobs.
Following through on this commitment, Microsoft last year got the device certified as protective eyewear, which allows workers in the field to not have to wear any other eye protection under the headset. Microsoft teamed up with construction company Trimble to build a hardhat accessory aimed at aiding communications at job sites.
After trying out Remote Assist, I can see why it would be nice to have in the field. The virtual display freed up both hands to fix the problems in front of me. Not dealing with a phone or a paper manual sped up the process. My boss’ ability to draw up notes or directions within my field of vision allowed him to be a more active participant and communicate better with me.
It will be interesting to see how much traction Microsoft can get with these first-line workers given the cost of HoloLens. The headset is priced at $3,000 for a developer version and $5,000 for a commercial edition. It’s hard to see every worker in a big factory being outfitted with one of these devices given the cost.
Layout works with both HoloLens and the fully immersive virtual reality headsets made by Microsoft partners like Lenovo, Samsung, HP and more. In my demo, I was tasked with transforming a hotel conference room into a functioning factory floor.
First, I placed a robotic arm onto the floor near a power supply. I trained my vision on a menu with several items I can choose, then I did the air tap motion to pick the arm and hold the pinch to drag it around the floor.
I had some difficulties with this app because I couldn’t always tell where exactly on the floor I was placing the model machines and struggled a little bit to place objects right where I wanted them. And a couple inches here and there can make a big difference.
Layout gave me a good sense of the room, with the area I was designing for taped off. I could make sure that things were spaced out right, no one was going to run into the machines and there weren’t any unanticipated issues. For example, the original design had a machine in one spot, but the control panel was lined up directly against a pillar that wasn’t in the blueprints. Layout gave me the option to rotate the item until the control panel was where I wanted it to be.
I found Remote Assist to be a more compelling tool than Layout. The ability to communicate hands free and have someone hundreds or thousands of miles away see through my eyes and give me notes in my field of vision stood out.
Layout will be really helpful for visual types who want to see what a room looks before committing to a plan. But I found it not quite as intuitive to use as Remote Assist.
Bottom line, these two apps figure to strengthen Microsoft’s pitch to make HoloLens a must-have for the first-line workers it covets. The company will have to continue to find new ways to help these workers do their jobs better to the point that it’s worth it for the higher-ups to spend $3,000 to $5,000 a pop to outfit the workforce.
