Unearthing Microsoft’s HoloLens feels a little like walking the decks of the Titanic. Three years ago, Microsoft’s augmented-reality headset ignited the imaginations of consumers and developers alike with its promise of lifelike animated sprites that could perch on real-world objects. Then… it sank.
It’s almost criminal that Microsoft’s original HoloLens demos never saw the light of day. Bending down to peer “inside” a coffee table into the Minecraft underworld was an utterly transformative experience. But at least Microsoft’s vision of using the HoloLens as a business tool apparently is alive and well.
As is the HoloLens. Because Microsoft never sold its device to consumers, PCWorld never formally reviewed it. But we have the original on hand. As Microsoft is expected to bring forth the next generation of HoloLens at Mobile World Congress (MWC) in Barcelona this weekend, we gave it another run-through, focusing on what made it best—and what could be better.
Mark Hachman / IDGIn this shot of the Microsoft HoloLens, you can see what Microsoft calls the “environment understanding cameras” that map the world around you, as well as the front-facing 2MP camera.
A consumer failure with corporate potential
As a consumer device, the HoloLens is dead. Only one Microsoft Store on the West Coast even has a HoloLens in stock for demonstration purposes. Microsoft’s remaining HoloLens supplies are sold through Insight, a solutions provider, Microsoft support reps said. On Microsoft’s site, the “Commercial Suite” version of the HoloLens costs a cool $5,000.
Mark Hachman / IDGThe HoloLens shipped in a hard plastic case.
But commercial it was not, back in 2015. Removing it from its cloth-covered plastic shell three years later reveals a lovingly engineered piece of consumer hardware. The HoloLens perches fairly comfortably on my head, supported by a removeable nose piece and my ears. Clicking a dial adjusts the tightness of the fit. There’s an optional head strap, too, if you’d prefer taking its 1.28-pound weight on the crown of your head. I wore the HoloLens pretty solidly for about three hours in one stretch. While I was conscious of the weight, I never felt uncomfortable.
Mark Hachman / IDGThe HoloLens audio is controlled by a pair of buttons on one side; you can control the brightness of the display with the others.
The HoloLens includes built-in speakers, which can be adjusted with a pair of buttons on one side. On the other, a similar pair of buttons controls the brightness of the display. An optional headphone jack is available too. Underneath the power button is a row of pinprick-sized holes, which light to indicate the battery level. (The HoloLens drains fast, and our unit provided about three to four hours of charge in fairly constant use.)
All that pales in comparison to the focal point of the HoloLens: the display. While virtual-reality devices like the first HTC Vive tethered you to the PC by way of a display and power cord, the augmented-reality HoloLens superimposes the display over the real world—understanding where physical surfaces are, and placing objects accordingly. You’re free to roam wherever you’d like. It’s magical, when it works.
Mark Hachman / IDGA dial ratchets the HoloLens tightly around your head. I preferred the weight (1.2) pounds to rest on my nose and the brim, and from there my ears. An optional head strap can take some of the weight, too.
By now, the hardware underlying the HoloLens is fairly well understood, at least in broad strokes. In terms of resolution, though, the lack of a PC-powered display means that the HoloLens essentially provides 720p of resolution per eye. Combine that with the relatively limited field of view, and that’s where the experience gets complicated.
The first fix: Field of View (FOV)
HoloLens works its magic by suspending holograms, or what appears to be a three-dimensional object, virtually in front of you. There’s not that much difference between the image that a monitor paints on the glass in front of your eyes and the image that the HoloLens paints on each lens in front of you.
Mark Hachman / IDGStanding about five feet away, the HoloLens field of view is about the size of this window. (The HoloLens front-facing camera is offset, with a slightly wider angle of view, so it captures more than you see.) The photos you’ll see here are shot with the HoloLens mixed-reality camera, overlaying holograms onto the real world.
The difference is what HoloLens does next. It uses a front-facing camera to scan the room in front of you, interpreting what it sees as a three-dimensional mesh, then attaches the hologram to that mesh itself. Using the HoloLens usually means anchoring various windows (such as Settings, for example) to a nearby surface, then using any three-dimensional open space near you as your HoloLens playspace. The illusion of physically walking around the hologram, as well as being able to anchor it or have it interact with real-world surfaces, fools your eyes into thinking it’s real.
Mark Hachman / IDGThe farther back you are, the more you can see. This is about eight feet away, against a different door.
That illusion is especially effective if the hologram remains in the field of view—the rectangle that serves as the HoloLens screen. The FOV covers much of the width of a door at a few feet, or about 50 degrees. Anything outside of it, though, can’t be entirely seen, and this instantly reminds you of the limits the HoloLens FOV imposes.
Mark Hachman / IDGBack up far enough, and even large holograms like this Tyrannosaurus Rex can fit into your field of view (FOV).
Because the HoloLens “knows” where in virtual space a hologram exists, it’s possible to have one directly in front of you and another to the side or rear. It’s trying to turn and find the second hologram when frustration can kick in. (A setting allows you to trash all holograms loaded into your scene, even the ones you can’t see.) But the first-generation HoloLens could never convincingly offer up the illusion of, say, a forest, as the HoloLens lacks the capability to swamp your peripheral vision.
The FOV problem isn’t just a visual issue. Interacting with holograms and apps typically requires “air clicking” a hologram or window, in much the same way you’d click a mouse. Triggering the Start menu requires a “bloom” hand gesture, like a flower opening. In both cases, though, the HoloLens must be able to detect your gesture. (There are two alternatives—voice control and a small Bluetooth “clicker”—but gestures are the most common way of interacting.)
Mark Hachman / IDGHolograms can either be “two-dimensional” Windows apps, or full-fledged three-dimensional objects. Using a two-handed gesture, you can place these windowed apps along surfaces, where HoloLens will remember them, even if you’re facing the other way. (The front-facing camera is only 2MP, so the resolution here is lousy. They look better in real life.)
Anyone who’s used the HoloLens, then, would rate the field-of-view as the primary area for improvement. If I were designing the next-generation HoloLens, however, I’d strike a compromise, offering users the option of dialing down the peripheral detail.
In the VR space, this is known as foveated rendering, concentrating the processing power on rendering detail where your eyes are looking. That’s probably a necessity for a new HoloLens, too—more on that in a minute. But I also appreciated the blank space at the edge of my vision, either to glance at my PC’s screen, a person, or something else. If Microsoft were to improve the HoloLens, I’d leave the option to “fade out” the periphery, providing a more grounded sense of place to the user.
The second fix: Resolution
It’s no secret that early VR users suffered from vertigo, caused by the latency between what the displays rendered and what the user “expected” to see. Though there’s still some noticeable render lag within the HoloLens, the fact that your vision tracks real-world objects helps prevent any nausea.
Mark Hachman / IDGIt’s a little easier to tell that’s actor George Takei when using the HoloLens, and an app called Actiongram. But if Microsoft is striving for realism, more resolution is a must.
I still experienced discomfort. Dialing up the brightness of the display is a must. Even then, however, holograms can be low-resolution, and sometimes laggy when interacting with you. After an hour or two, my eyes ached from the strain. A reported processor spec bump must happen to ensure a next-gen HoloLens will be comfortable to use.
The resolution isn’t merely a visual issue, either. HoloLens maps your environment, applying a very coarse surface mesh over walls, desks, tables, and more. (Objects can be “anchored” to a surface like a desk, and certain games will actually use these surfaces as part of the experience.) A spartan workspace is easily accommodated. But as your environment moves further away from minimalism, HoloLens struggles to keep up.
My basement office, full of monitors and boxes and peripherals and lamps, would probably give organizational guru Marie Kondo the cold sweats. To compensate, HoloLens would occasionally draw a big black surface that would interrupt the scene. A next-gen HoloLens would be well served by increasing the number and density of the three-dimensional mesh of points it uses to creates a virtual space.
The third fix: UI
Certain elements of the HoloLens UI haven’t held up over time, including the “nose first” method of navigating. Too much navigation involves steering a cursor from point to point by slightly moving your head. It’s especially painful when you have to enter a Windows or Wi-Fi password: Manually pushing the cursor from letter to letter is a major pain.
Mark Hachman / IDGNavigating the Start menu through the bloom gesture is fine. But when you have to navigate, letter by letter, to spell out a password—ugh. (Again, the HoloLens camera is awful, but there’s no way to take a photo without it.)
Microsoft’s first-generation HoloLens works around this somewhat through voice recognition, where many simple tasks, such as opening the Settings menu, can be performed by merely saying a word, like “settings.” But modern VR equipment is beginning to include eye tracking, which can save time navigating from element to element. If Microsoft is developing a next-generation HoloLens, it would be a shame if eye tracking were excluded.
The HoloLens voice recognition works well, but even it can be unnecessarily confusing. Some commands can be performed simply by saying the command; others require you to invoke Cortana beforehand, which makes little sense. Still, I found I relied on voice more and more to avoid tediously steering the cursor.
Mark Hachman / IDGThis little gizmo is the HoloLens “clicker,” which you can click to input choices, or “twist” to scroll. The important aspect of the clicker is that you can click without the HoloLens camera seeing your movements, which is far more comfortable than simply air-clicking all of the time.
Lifting your hand to constantly “air click” gets old, too. But scrolling is the worst, requiring you to air-click and drag through the minimal field of view. Microsoft’s optional “clicker” basically acknowledged that air navigation failed. But couldn’t Microsoft have tried something a bit different, such as “clicking” by audibly snapping your fingers?
The fourth fix: More apps
If you think the Windows app store looks empty, you should see the HoloLens aisle. At one point, UWP apps like Twitter were considered to be the future, traversing PC, phone, and HoloLens alike.
Mark Hachman / IDGIt’s just another hologram, but opening the Rome tour on Microsoft Studios’ own HoloTour via this painting is a great start, especially as it seamlessly works its way into…
You can exhaust most of the best HoloLens apps in an afternoon. HoloTour offers genuinely interesting 360-degree recorded video tours of Machu Picchu and Rome, complete with a recorded tour guide. Step back, though, and they aren’t much different from a HoloLens-specific version of Myst, as you virtually explore hotspot after hotspot. Holo Galaxy isn’t much better, with a cursory 3D view of the solar system and other stellar points of interest.
Mark Hachman / IDG…A 360-degree video tour of Rome. Yes, you’ve probably seen 360-degree video before. But it’s all nicely presented, with a friendly, informative narrator.
RoboRaid, jaw-dropping in its time, creates virtual bugs and drones that literally crawl out from your HoloLens-mapped walls. You’ll need to use oral commands, your air-click skills, and the ability to hop nimbly about the room to avoid laser blasts. It’s still a lot of fun, but not too different from the “shooting gallery” games that tend to be bundled with VR gear.
Mark Hachman / IDGIn RoboRaid, robotic bugs and drones crawl out of the walls, firing laser blasts, which you must physically avoid either by ducking or moving out of the way. Air clicking returns fire.
