Why Virtual Reality and Augmented Reality are the same thing – Computerworld

Words matter. And as a stickler for accuracy in language that describes technology, it pains me to write this column.

I hesitate to expose the truth, because the public is already confused about virtual reality (VR), augmented reality (AR), mixed reality (MR), 360-degree video and heads-up displays. But facts are facts. And the fact is that the technology itself undermines clarity in language to describe it.

Before we get to my grand thesis, let's kill a few myths.

Silicon Valley just produced a mind-blowing new virtual reality product. It's a sci-fi backpack that houses a fast computer to power a high-resolution VR headset. Welcome to the future of VR gaming, right?

Wrong.

While the slightly-heavier-than-10-pound backpack is conceptually similar to existing gaming rigs, it's actually designed for enterprises, as well as healthcare applications. It's called the Z VR Backpack from HP. It works either with HP's new Windows Mixed Reality Headset or with HTC's Vive business edition headset, and houses a Windows 10 Pro PC, complete with an Intel Core i7 processor, 32GB of RAM and, crucially, an Nvidia Quadro PS2000 graphics card. It also has hot-swappable batteries.

Will HP's new enterprise-ready VR backpack deliver mixed reality, augmented reality or virtual reality? The answer is yes!

To me, the biggest news is that HP plans to open 13 customer experience centers around the world to showcase enterprise and business VR applications. If that surprises you, it's because the narrative around VR is that it's all about immersive gaming and other "fun" applications. It's far more likely that professional uses for VR will dwarf the market for consumer uses.

All of these technologies have been around for decades, at least conceptually. Just now, on the brink of mainstream use for both consumer and business applications, it's important to recognize that different people mean different things when they use the labels to describe these new technologies.

A Singapore-based company called Yi Technology this week introduced an apparently innovative mobile gadget called the Yi 360 VR Camera. The camera takes 5.7k video at 30 frames per second, and is capable of 2.5k live streaming.

Impressive! But is 360-degree video "virtual reality"? Some (like Yi) say yes. Others say no. (The correct answer is "yes" more on that later.)

Mixed reality and augmented reality are also contested labels. Everyone agrees that both mixed reality and augmented reality describe the addition of computer-generated objects to a view of the real world.

One opinion about the difference is that mixed reality virtual objects are "anchored" in reality they're placed specifically, and can interact with the real environment. For example, mixed reality objects can stand on or even hide behind a real table.

By contrast, augmented reality objects are not "anchored," but simply float in space, anchored not to physical spaces but instead to the user's field of view. That means Hololens is mixed reality, but Google Glass is augmented reality.

People disagree.

An alternative definition says that mixed reality is a kind of umbrella term for virtual objects placed into a view of the real world, while augmented reality content specifically enhances the understanding of, or "augments," reality. For example, if buildings are labeled or people's faces are recognized and information about them appears when they're in view, that's augmented reality in this definition.

Under this differentiation, Google Glass is neither mixed nor augmented reality, but simply a heads-up display information in the user's field of view that neither interacts with nor refers to real-world objects.

Complicating matters is that the "mixed reality" label is falling out of favor in some circles, with "augmented reality" serving as the umbrella term for all technologies that combine the real with the virtual.

If the use of "augmented reality" bothers you, just wait. That, too, may soon become unfashionable.

And now we get to the confusing bit. Despite clear differences between some familiar applications of, say, mixed reality and virtual reality, other applications blur the boundaries.

Consider new examples on YouTube.

One video shows an app built with Apple's ARKit, where the user is looking at a real scene, with one computer-generated addition: A computer-generated doorway in the middle of the lane creates the illusion of a garden world that isn't really there. The scene is almost entirely real, with one door-size virtual object. But when the user walks through the door, they are immersed in the garden world, and can even look back to see the doorway to the real world. On one side of the door, it's mixed reality. On other side, virtual reality. This simple app is MR and VR at the same time.

A second example is even more subtle. I'm old enough to remember a pop song from the 1980s called Take On Me by a band called A-ha. In the video, a girl in a diner gets pulled into a black-and-white comic book. While inside, she encounters a kind of window with "real life" on one side and "comic book world" on the other.

Someone explicitly created an app that immerses the user in a scenario identical to the "A-ha" video, wherein a tiny window gives a view into a charcoal-sketch comic world clearly "mixed reality" but then the user can step into that world, entering a fully virtual environment, with the exception of a tiny window into the real world.

This scenario is more semantically complicated than the previous one because all the "virtual reality" elements are in fact computer-modified representations of real-world video. It's impossible to accurately describe this app using either "mixed reality" or "virtual reality."

When you look around and see a live, clear view of the room you're in, that's 360-degree video, not virtual reality. But what if you see live 360 video of a room you're not in one on the other side of the world? What if that 360 video is not live, but essentially recorded or mapped as a virtual space? What if your experience of it is like you're tiny, like a mouse in a giant house, or like a giant in a tiny house? What if the lights are manipulated, or multiple rooms from different houses stitched together to create the illusion of the same house? It's impossible to differentiate at some point between 360 video and virtual reality.

Purists might say live, 360 video of, say, an office, is not VR. But what if you change the color of the furniture in software? What if the furniture is changed in software to animals? What if the walls are still there, but suddenly made out of bamboo? Where does the "real" end and the "virtual" begin?

Ultimately, the camera that shows you the "reality" to be augmented is merely a sensor. It can show you what you would see, along with virtual objects in the room, and everybody would be comfortable calling that mixed reality. But what if the app takes the motion and distance data and represents what it sees in a changed form. Instead of your own hands, for example, it could show robot hands in their place, synchronized to your actual movement. Is that MR or VR?

The next version of Apple maps will become a kind of VR experience. You'll be able to insert an iPhone into VR goggles and enter 3D maps mode. As you turn your head, you'll see what a city looks like as if you were Godzilla stomping through the streets. Categorically, what is that? (The 3D maps are "computer generated," but using photography.) It's not 360 photography.

The "blending" of virtual and augmented reality is made possible by two facts. First, all you need is a camera lashed to VR goggles in order to stream "reality" into a virtual reality scenario. Second, computers can augment, modify, tweak, change and distort video in real time to any degree desired by programmers. This leaves us word people confused about what to call something. "Video" and "computer generated" exist on a smooth spectrum. It's not one or the other.

This will be especially confusing for the public later this year, because it all goes mainstream with the introduction of the iPhone 8 (or whatever Apple will call it) and iOS 11, both of which are expected to hit the market within a month or two.

The Apple App Store will be flooded with apps that will not only do VR, AR, MR, 360 video and heads-up display content (when the iPhone is inserted into goggles) but that will creatively blend them in unanticipated combinations. Adding more confusion, some of the most advanced platforms, such as Microsoft Hololens, Magic Leap, Meta 2, Atheer AiR and others, will not be capable of doing virtual reality.

Cheap phones inserted into cardboard goggles can do VR and all the rest. But Microsoft's Hololens cannot.

All these labels are still useful for describing most of these new kinds of media and platforms. Individual apps may in fact offer mixed reality or virtual reality exclusively.

Over time we'll come to see these media in a hierarchy, with heads-up displays at the bottom and virtual reality at the top. Heads-up display devices like Google Glass can do only that. But "mixed reality" platforms can do mixed reality, augmented reality and heads-up display. "Virtual reality" platforms (those with cameras attached) can do it all.

Word meanings evolve and shift over time. At first, alternative word use is "incorrect." Then it's acceptable in some circles, but not others. Eventually, if enough people use the formerly wrong usage, it becomes right. This is how language evolves.

A great example is the word "hacker." Originally, the word referred to an "enthusiastic and skilful computer programmer or user." Through widespread misuse, however, the word has come to primarily mean "a person who uses computers to gain unauthorized access to data."

Prescriptivists and purists argue that the old meaning is still primary or exclusive. But it's not. A word's meaning is decided by how a majority of people use it, not by rules, dictionaries or authority.

I suspect that over time the blurring of media will confuse the public into calling VR, AR, MR, 360 video and heads-up display "virtual reality" as the singular umbrella term that covers it all. At the very least, all these media will be called VR if they're experienced through VR-capable equipment.

And if we're going to pick an umbrella term, that's the best one. It's still close enough to describe all these new media. And in fact only VR devices can do it all.

Welcome to the fluid, flexible multimedia world of heads-up display, 360 video, mixed reality, augmented reality and virtual reality.

It's all one world now. It's all one thing. Just call it "virtual reality."