Oculus’ New Experimental API Blends Virtual Reality With Your Real-World Surroundings – Gizmodo

Photo: Josh Edelson (Getty Images)

An update for Facebooks Oculus Quest 2 virtual reality system will let developers incorporate real-life video from the VR headsets sensorsinto their games to create mixed reality experiences.

With Passthrough API Experimental, Oculus new application programming interface, developers can customize how a players surroundings appear through their VR headset, applying effects and filters and even rendering the real world onto certain in-game surfaces. The API will roll out first to Unity developers in an upcoming software update, with support for other development platforms coming in the future, Oculus VR said in a blog post Friday.

Its easy to imagine a slew of cool ways games could incorporate your physical environment into gameplay. One gif that Oculus shared of the API in action shows a player drawing on the walls, immediately calling to mind the paint-based turf wars a la Splatoon.

Gif: Oculus VR (Facebook)

The ability to toggle the mixed realitys opacity, aka how much of the real-world or virtual one you see at any given time, thats shown off in another gif could handily be integrated with some kind of puzzle-solving mechanic.

G/O Media may get a commission

In-game enemies could also hide behind your furniture for sneak attacks. Im a total chicken when it comes to horror games already, so the idea of monsters being able to jump out from behind my own living room couch makes me want to cry.

Gif: Oculus VR (Facebook)

The announcement included several examples of uses for the API beyond gaming as well. Oculus posited that it could improve productivity and enable more collaborative teleworking by incorporating workers real-life keyboards and desks. Users would also be able to engage with virtual content without losing the ability to interact with their housemates or pets.

When asked if first-generation Quest users can expect to have access, Facebook told the folks at UploadVR that the API is only coming to the Quest 2. A pared-down version of the Passthrough technology is already available on Facebooks Quest, Quest 2, and Rift S headsets, which lets users take a peek at whats happening around them while still wearing the headset.

Gif: Oculus VR (Facebook)

In its announcement, Oculus added that it designed the API with privacy in mind.

Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device, the company said.

As for when developers can expect to ship their games that use Passthrough to players, Oculus said its aiming to roll out a production version later this year.

Read more:

Oculus' New Experimental API Blends Virtual Reality With Your Real-World Surroundings - Gizmodo