For the last half-decade at each Oculus Connect, Facebooks top VR researcher presented an annual look at the future of the technology.

The research-focused talk by Michael Abrash is a highlight of the annual conference hosted by Facebook and well hope to see a similar update during Oculus Connect 6. Hired from Valve, Abrash built up Facebooks long-term VR research efforts first at Oculus Research and then under its new name Facebook Reality Labs.

You can watch everything he said about the future of the technology during his presentations from 2014 to 2018 in the video below.

Abrash offers an overview of Oculus Research and tries addressing the question of with VR failing to reach mass adoption in the past why it is going to be different this time.

In a very real sense its the final platform, he says. The one which wraps our senses and will ultimately be able to deliver any experience that were capable of having.

He says Oculus Research is the first well-funded VR research team in 20 years and their job is to do the deep, long-term work of advancing the VR platform. He points to a series of key areas they plan to pursue including eye tracking. The idea behind foveated rendering is that if you track the eyeballs movements fast and reliably enough you could build a VR headset which only draws the most detailed parts of a scene directly where you are looking. He also described the fixed focal depth of modern VR headsets as not perceptually ideal and admits they can cause discomfort or may make VR subtly less real while hinting at several possible ways of addressing this problem requiring new hardware and changes to the rendering model.

This is what it looks like when opportunity knocks, he says.

In late 2015 Abrash outlines a more specific series of advances required to drive human senses with VR technology. He says hes fine leaving the sense of taste to future VR researchers, and both touch and smell require the development of breakthroughs in delivery techniques. He also discusses the vestibular system which he describes as our internal accelerometer and gyroscope for sensing change in orientation and acceleration and that conflict between our vestibular sense and what you see is a key cause of discomfort.

Right now theres no traction on the problem, he says.

For hearing, though, theres a clear path to doing it almost perfectly, he says. Clear doesnt mean easy though. He breaks down three elements of audio simulation as Synthesis (the creation of source sounds), Propagation (how sound moves around a space), and Spatialization (the direction of incoming sound) and the difficulties involved in doing all three well.

We understand the equations that govern sound but were orders of magnitude short of being able to run a full simulation in real time even for a single room with a few moving sound sources and objects, he says.

He predicts that in 20 years youll be able to hear a virtual pin drop and it will sound right the interesting question is how close well be able to get in five years.

He describes photon delivery systems as needing five attributes which are often in conflict with one another, requiring trade-offs in field of view, image quality, depth of focus, high dynamic range and all-day ergonomics.

All currently known trade offs are a long way from real-world vision, he says.

He provides the following chart showing the current market standard and the desired attributes of a future VR vision system.

Abrash describes other areas of intense interest for Facebooks VR research as scene reconstruction as well as body tracking and human reconstruction. Interaction and the development of dexterous finger control for VR is a particularly difficult problem to solve, he adds.

Theres no feasible way to fully reproduce real-world kinematics, he says. Put another way, when you put your hand down on a virtual table theres no known or prospective consumer technology that would keep your hand from going right through it.

He says it is very early days and the first haptic VR interface that really works will be world-changing magic on par with the first mouse-based windowing systems.

Facebooks PC-powered Oculus Rift VR headset launched in 2016 but Facebook hadnt yet shipped the Oculus Touch controllers for them just yet. Echoing the comments he made in 2015, he says haptic and kinematic technology that isnt even on the distant horizon is needed to enable the use of your hands as direct physical manipulators. As a result, he says, Touch-like controllers will still be the dominant mode for sophisticated VR interactions in five years.

The prediction came in a series as Abrash essentially outlines what a new PC-powered Rift made in 2021 might be able to achieve. The biggest risk to many of his predictions, though, is that eye-tracking quality required for many advances in VR displays is not a solved problem. It is central to the future of VR, he says. He suggests foveated rendering is even key to making a wireless PC VR headset work.

Eliminating the tether will allow you to move freely about the real world while in VR yet still have access to the processing power of a PC, he said.

He said he believes virtual humans will still exist in the uncanny valley and that convincingly human avatars will be longer than five years away.

At Oculus Connect 4 in 2017 Facebook changed the Abrash update into a conversational format. Among the questions raised for Abrash was how his research teams contribute to VR products at the company.

Theres nothing in the current generation that has come from us, he said. But there is certainly a number of things that we could see over the next few years.

While there isnt a lot about the future in this session the following comment helped explain what he sees as the purpose of Facebooks VR and AR research:

How do we get photons into your eyes better, how do we give you better computer vision for self-presence, for other peoples presence, for the surroundings around you. How do we do audio better, how do we let you interact with the world better it is a whole package and each piece can move forward some on its own but in the long run you really want all the pieces to come together. And one really good example is suppose that we magically let you use your hands perfectly in VR, right? You just reach out, you grab virtual objects well remember that thing I said about where youre focused? Everything within hands length wouldnt actually be very sharp and well focused right. So you really to solve that problem too. And it just goes on and on like that where you need all these pieces to come together in the right system and platform.

At last years Oculus Connect conference Abrash updated some of his predictions from 2016. While they were originally slated for arrival in high-end VR headsets in 2021, this time he says he thinks theyll likely be in consumer hands by 2022.

He suggests in this presentation that the rate of advancement in VR is ramping up faster than he predicted thanks to the parallel development of AR technology.

He suggests new lenses and waveguide technology might have a huge impact on future display systems. He also says that foveated rendering and great eye tracking still represent a risk to his predictions, but now hes comfortable committing to a prediction that highly reliable eye tracking and foveated rendering should be doable by 2022.

Audio presence is a real thing, Abrash said of Facebooks sound research. It may take longer than I thought.

Abrash showed Facebooks work on codec avatars for convincing human avatar reconstructions and suggests that it is possible these might arrive along the same time frame as his other predictions 2022.

He also makes his longest-term prediction in saying he believes by the year 2028 well have useful haptic hands in some form.

View post:

Oculus Chief Researcher's Predictions Of The Future Of Virtual Reality - UploadVR