by Chris Woodford. Last updated: March 3, 2017.

You'll probably never go to Mars, swim with dolphins, run anOlympic 100 meters, or sing onstage with the Rolling Stones. But ifvirtual reality ever lives up to its promise, you might be able to doall these thingsand many morewithout even leaving your home.Unlike real reality (the actual world in which we live),virtual reality means simulating bits of our world (or completelyimaginary worlds) using high-performance computers and sensoryequipment, like headsets and gloves. Apart from games andentertainment, it's long been used for training airline pilots andsurgeons and for helping scientists to figure out complex problemssuch as the structure of protein molecules. How does it work? Let's take acloser look!

Photo: Virtual reality means blocking yourself off from the real world and substitutinga computer-generated alternative. Often, it involves wearing a wraparound headset called a head-mounted display, clamping stereo headphones over your ears, and touching or feeling your way around your imaginary home using datagloves (gloves with built-in sensors). Picture by Wade Sisler courtesy of NASA Ames Research Center.

Virtual reality (VR) means experiencing things through ourcomputers that don't really exist. From that simple definition, theidea doesn't sound especially new. When you look at an amazingCanaletto painting, for example, you're experiencing the sites andsounds of Italy as it was about 250 years agoso that's a kind ofvirtual reality. In the same way, if you listen to ambientinstrumental or classical music with your eyes closed, and startdreaming about things, isn't that an example of virtual realityanexperience of a world that doesn't really exist? What about losingyourself in a book or a movie? Surely that's a kind of virtualreality?

If we're going to understand why books, movies, paintings, andpieces of music aren't the same thing as virtual reality, we need todefine VR fairly clearly. For the purposes of this simple, introductory article,I'm going to define it as:

Putting it another way, virtual reality is essentially:

Artwork: This Canaletto painting of Venice, Italy is believable and in some sense explorable (you can move your eyes around and think about different parts of the picture), but it's not interactive, computer-generated, or immersive, so it doesn't meet our definition of virtual reality: looking at this picture is not like being there. There's nothing to stop us making an explorable equivalent in VR, but we need CGInot oil paintsto do it. Picture courtesy of Wikimedia Commons.

We can see from this why reading a book, looking at a painting,listening to a classical symphony, or watching a movie don't qualifyas virtual reality. All of them offer partial glimpses ofanother reality, but none are interactive, explorable, or fullybelievable. If you're sitting in a movie theater looking at a giantpicture of Mars on the screen, and you suddenly turn your head toofar, you'll see and remember that you're actually on Earth and theillusion will disappear. If you see something interesting on thescreen, you can't reach out and touch it or walk towards it; again,the illusion will simply disappear. So these forms of entertainmentare essentially passive: however plausible they might be, theydon't actively engage you in any way.

VR is quite different. It makes you think you are actually livinginside a completely believable virtual world (one in which, to usethe technical jargon, you are partly or fully immersed). It istwo-way interactive: as you respond to what you see, what you seeresponds to you: if you turn your head around, what you see or hearin VR changes to match your new perspective.

"Virtual reality" has often been used as a marketing buzzwordfor compelling, interactive video games or even 3D movies andtelevision programs, none of which really count as VR because they don't immerseyou either fully or partially in a virtual world. Search for "virtualreality" in your cellphone app store and you'll find hundreds ofhits, even though a tiny cellphone screen could never get anywherenear producing the convincing experience of VR. Nevertheless, thingslike interactive games and computer simulations would certainly meetparts of our definition up above, so there's clearly more thanone approach to building virtual worldsand more than one flavor ofvirtual reality. Here are a few of the bigger variations:

For the complete VR experience, we need three things. First, aplausible, and richly detailed virtual world to explore; a computer modelor simulation, in other words. Second, a powerful computer thatcan detect what we're going and adjust our experience accordingly, inreal time (so what we see or hear changes as fast as we movejustlike in real reality). Third, hardware linked to the computer thatfully immerses us in the virtual world as we roam around. Usually,we'd need to put on what's called a head-mounted display (HMD) withtwo screens and stereo sound, and wear one or more sensory gloves.Alternatively, we could move around inside a room, fitted out withsurround-sound loudspeakers, onto which changing images are projectedfrom outside. We'll explore VR equipment in more detail in a moment.

A highly realistic flight simulator on a home PC might qualify asnonimmersive virtual reality, especially if it uses a very widescreen, with headphones or surround sound, and a realistic joystickand other controls. Not everyone wants or needs to be fully immersedin an alternative reality. An architect might build a detailed 3Dmodel of a new building to show to clients that can be explored on adesktop computer by moving a mouse. Most people would classify thatas a kind of virtual reality, even if it doesn't fully immerse you.In the same way, computer archaeologists often create engaging 3Dreconstructions of long-lost settlements that you can move around andexplore. They don't take you back hundreds or thousands of years orcreate the sounds, smells, and tastes of prehistory, but they give amuch richer experience than a few pastel drawings or even an animatedmovie.

What about "virtual world" games like Second Life and Minecraft? Do theycount as virtual reality? Although they meet the first four of ourcriteria (believable, interactive, computer-created and explorable),they don't really meet the fifth: they don't fully immerse you. Butone thing they do offer that cutting-edge VR typically doesn't iscollaboration: the idea of sharing an experience in a virtualworld with other people, often in real time or something very closeto it. Collaboration and sharing are likely to become increasinglyimportant features of VR in future.

Virtual reality was one of the hottest, fastest-growingtechnologies in the late 1980s and early 1990s, but the rapid rise ofthe World Wide Web largely killed off interest after that. Eventhough computer scientists developed a way of building virtual worldson the Web (using a technology analogous to HTML called VirtualReality Markup Language, VRML), ordinary people were much moreinterested in the way the Web gave them new ways to access realrealitynew ways to find and publish information, shop, and sharethoughts, ideas, and experiences with friends through social media.With Facebook's growing interest in the technology, the future of VRseems likely to be both Web-based and collaborative.

Photo: Augmented reality: A heads-up display, like this one used by the US Air Force,superimposes useful, computer-based information on top of the things you see with your own eyes. Picture by Major Chad E. Gibson courtesy of US Air Force.

Mobile devices like smartphones and tablets have put what used tobe supercomputer power in our hands and pockets. If we're wandering round the world, maybe visiting a heritage site like the pyramids or a fascinatingforeign city we've never been to before, what we want is typicallynot virtual reality but an enhanced experience of the excitingreality we can see in front of us. That's spawned the idea ofaugmented reality (AR), where,for example, you point your smartphone at alandmark or a striking building and interesting information about itpops up automatically. Augmented reality is all about connecting thereal world we experience to the vast virtual world of informationthat we've collectively created on the Web. Neither of these worldsis virtual, but the idea of exploring and navigating the twosimultaneously does, nevertheless, have things in common with virtualreality. For example, how can a mobile device figure out its preciselocation in the world? How do the things you see on the screen ofyour tablet change as you wander round a city? Technically, theseproblems are similar to the ones developers of VR systems have tosolveso there are close links between AR and VR.

Close your eyes and think of virtual reality and you probablypicture something like our top photo: a geek wearing a wraparoundheadset (HMD) and datagloves, wired into a powerful workstation orsupercomputer. What differentiates VR from an ordinary computerexperience (using your PC to write an essay or play games) is thenature of the input and output. Where an ordinary computer usesthings like a keyboard,mouse, or (more exotically)speech recognition for input, VR uses sensors that detect how your body ismoving. And where a PC displays output on a screen (or a printer), VRuses two screens (one for each eye), stereo or surround-soundspeakers, and maybe some forms of haptic (touch and body perception)feedback as well. Let's take a quick tour through some of the morecommon VR input and output devices.

Photo: The view from inside. A typical HMD has two tiny screensthat show different pictures to each of your eyes, so your brain produces a combined3D (stereoscopic) image. Picture by courtesy of US Air Force.

There are two big differences between VR and looking at anordinary computer screen: in VR, you see a 3D image that changessmoothly, in real-time, as you move your head. That's made possibleby wearing a head-mounted display, which looks like a giant motorbikehelmet or welding visor, but consists of two small screens (one infront of each eye), a blackout blindfold that blocks out all otherlight (eliminating distractions from the real world), and stereoheadphones. The two screens display slightly different, stereoscopicimages, creating a realistic 3D perspective of the virtual world.HMDs usually also have built-in accelerometers or position sensorsso they can detect exactly how your head and body are moving (bothposition and orientationwhich way they're tilting or pointing) andadjust the picture accordingly. The trouble with HMDs is that they'requite heavy, so they can be tiring to wear for longperiods; some of the really heavy ones are even mounted on standswith counterweights. But HMDs don't have to be so elaborateand sophisticated: at the opposite end of the spectrum, Googlehas developed an affordable, low-cost pair of cardboard goggleswith built-in lenses that convert an ordinary smartphone into a crude HMD.

An alternative to putting on an HMD is to sit or stand inside aroom onto whose walls changing images are projected from outside. As youmove in the room, the images change accordingly. Flight simulatorsuse this technique, often with images of landscapes, cities, andairport approaches projected onto large screens positioned justoutside a mockup of a cockpit. A famous 1990s VR experiment calledCAVE (Cave Automatic Virtual Environment), developed at theUniversity of Illinois by Thomas de Fanti, also worked this way.People moved around inside a large cube-shaped room withsemi-transparent walls onto which stereo images were back-projectedfrom outside. Although they didn't have to wear HMDs, they did needstereo glasses to experience full 3D perception.

See something amazing and your natural instinct is to reach outand touch iteven babies do that. So giving people the ability tohandle virtual objects has always been a big part of VR. Usually,this is done using datagloves, which are ordinary gloves with sensorswired to the outside to detect hand and figure motions. One technicalmethod of doing this uses fiber-optic cables stretched the length ofeach finger. Each cable has tiny cuts in it so, as you flex yourfingers back and forth, more or less light escapes. A photocell atthe end of the cable measures how much light reaches it and thecomputer uses this to figure out exactly what your fingers are doing.Other gloves use strain gauges, piezoelectric sensors, orelectromechanical devices (such as potentiometers) to measure fingermovements.

Photos: Left/above: EXOS datagloves produced by NASA in the 1990s had very intricate external sensorsto detect finger movements with high precision. Picture courtesy of NASA Marshall Space Flight Center (NASA-MSFC).Right/below: This more elaborate EXOS glove had separate sensors on each finger segment, wired up to a single ribboncable connected up to the main VR computer. Picture by Wade Sisler courtesy of NASA Ames Research Center.

Artwork: How a fiber-optic dataglove works. Each finger has a fiber-optic cable stretched along its length. (1) At one end of the finger, a light-emitting diode (LED) shines light into the cable. (2) Light rays shoot down the cable, bouncing off the sides. (3) There are tiny abrasions in the top of each fiber through which some of the rays escape. The more you flex your fingers, the more light escapes. (4) The amount of light arriving at a photocell at the end gives a rough indication of how much you're flexing your finger. (5) A cable carries this signal off to the VR computer. This is a simplified version of the kind of dataglove VPL patented in 1992, and you'll find the idea described in much more detail in US Patent 5,097,252.

Even simpler than a dataglove, a wand is a stick you can use totouch, point to, or otherwise interact with a virtual world.It has position or motion sensors (such as accelerometers)built in, along with mouse-like buttons or scroll wheels. Originally,wands were clumsily wired into the main VR computer; increasingly,they're wireless.

Photo: A typical handheld virtual reality controller (complete with elastic bands), looking not so different from a video game controller. Photo courtesy of NASA Ames Research Center.

VR has always suffered from the perception that it's little morethan a glorified arcade gameliterally a "dreamy escape" fromreality. In that sense, "virtual reality" can be an unhelpfulmisnomer; "alternative reality," "artificial reality," or"computer simulation" might be better terms. Thekey thing to remember about VR is that it really isn't a fad orfantasy waiting in the wings to whistle people off to alternativeworlds; it's a hard-edged practical technology that's been routinelyused by scientists, doctors, dentists, engineers, architects,archaeologists, and the military for about the last 30 years. Whatsorts of things can we do with it?

Photo: Flight training is a classic application of virtual reality, though it doesn't use HMDs or datagloves. Instead, you sit in a pretend cockpit with changing images projected onto giant screens to give an impression of the view you'd see from your plane. The cockpit is a meticulous replica of the one in a real airplane with exactly the same instruments and controls. Photo by Javier Garcia courtesy of US Air Force.

Difficult and dangerous jobs are hard to train for. How can yousafely practice taking a trip to space, landing a jumbo jet, making aparachute jump, or carrying out brain surgery? All these things areobvious candidates for virtual reality applications. As we've seenalready, flight cockpit simulators were among the earliest VRapplications; they can trace their history back to mechanicalsimulators developed by Edwin Link in the 1920s.Just like pilots, surgeons are now routinely trained using VR. In a2008 study of735 surgical trainees from 28 different countries, 68 percent saidthe opportunity to train with VR was "good" or "excellent"for them and only 2 percent rated it useless or unsuitable.

Anything that happens at the atomic or molecular scale iseffectively invisible unless you're prepared to sit with your eyesglued to an electron microscope. But suppose you want to design newmaterials or drugs and you want to experiment with the molecularequivalent of LEGO. That's another obvious application for virtualreality. Instead of wrestling with numbers, equations, ortwo-dimensional drawings of molecular structures, you can snapcomplex molecules together right before your eyes. This kind of workbegan in the 1960s at the University of North Carolina at ChapelHill, where Frederick Brooks launchedGROPE, a project to develop a VR system for exploring the interactions between protein moleculesand drugs.

Photo: If you're heading to Mars, a trip in virtual reality could help you visualize what you'll find when you get there. Picture courtesy of NASA Ames Research Center.

Apart from its use in things like surgical training and drug design,virtual reality also makes possible telemedicine (monitoring,examining, or operating on patients remotely). A logical extension ofthis has a surgeon in one location hooked up to a virtual realitycontrol panel and a robot in another location (maybe an entirecontinent away) wielding the knife. The best-knownexample of this is the daVinci surgical robot, released in 2009, ofwhich several thousand have now been installed in hospitalsworldwide. Introduce collaboration and there's the possibility of awhole group of the world's best surgeons working together on aparticularly difficult operationa kind of WikiSurgery, if youlike!

Architects used to build models out of card and paper; now they'remuch more likely to build virtual reality computer models you canwalk through and explore. By the same token, it's generally muchcheaper to design cars, airplanes, and other complex, expensivevehicles on a computer screen than to model them inwood, plastic, orother real-world materials. This is an area where virtual realityoverlaps with computer modeling: instead of simply making animmersive 3D visual model for people to inspect and explore, you'recreating a mathematical model that can be tested for its aerodynamic,safety, or other qualities.

From flight simulators to race-car games, VR has long hovered onthe edges of the gaming worldnever quite good enough torevolutionize the experience of gamers, largely due to computersbeing too slow, displays lacking full 3D, and the lack of decent HMDsand datagloves. All that may be about to change with the developmentof affordable new peripherals like the Oculus Rift.

Like any technology, virtual reality has both good and bad points.How many of us would rather have a complex brain operation carriedout by a surgeon trained in VR, compared to someone who has merelyread books or watched over the shoulders of their peers? How many ofus would rather practice our driving on a car simulator before we setfoot on the road? Or sit back and relax in a Jumbo Jet, confident inthe knowledge that our pilot practiced landing at this very airport,dozens of times, in a VR simulator before she ever set foot in a realcockpit?

Critics always raise the risk that people may be seduced byalternative realities to the point of neglecting their real-worldlivesbut that criticism has been leveled at everything from radioand TV to computer games and the Internet. And, at some point, itbecomes a philosophical and ethical question: What is real anyway?And who is to say which is the better way to pass your time? Likemany technologies, VR takes little or nothing away from the realworld: you don't have to use it if you don't want to.

The promise of VR has loomed large over the world of computing forat least the last quarter centurybut remains largely unfulfilled.While science, architecture, medicine, and the military all rely onVR technology in different ways, mainstream adoption remainsvirtually nonexistent; we're not routinely using VR the way we usecomputers, smartphones, or the Internet. But the 2014 acquisition ofVR company Oculus, by Facebook, greatly renewed interest in the areaand could change everything. Facebook's basic idea is to let peopleshare things with their friends using the Internet and the Web. Whatif you could share not simply a photo or a link to a Web article butan entire experience? Instead of sharing photos of your wedding withyour Facebook friends, what if you could make it possible for peopleto attend your wedding remotely, in virtual reality, in perpetuity?What if we could record historical events in such a way that peoplecould experience them again and again, forever more? These are thesorts of social, collaborative virtual reality sharing that (we mightguess) Facebook is thinking about exploring right now. If so, thefuture of virtual reality looks very bright indeed!

So much for the future, but what of the past. Virtual reality hasa long and very rich history. Here are a few of the more interestinghighlights...

Artwork: The first virtual reality machine? Morton Heilig's 1962 Sensorama. Picture courtesy US Patent and Trademark Office.

Excerpt from:

What is virtual reality? - A simple introduction