Hands-on: Varjo Reality Cloud is a Platform for Capturing & Sharing Physical Spaces in Real-time – Road to VR

Varjo today announced its new Varjo Reality Cloud platform, a device-agnostic VR meeting platform which can scan and share physical spaces in real-time using the depth sensors on its latest XR-3 headsets. The company doesnt have a specific release date for the tech just yet, but is presenting a glimpse of what it plans to deliver in the future.Road to VR got an early look at the new tech.

Varjo is building a virtual meeting platform not unlike weve seen before, but with one key difference: the company plans to leverage the wide field-of-view depth sensors on its Varjo XR-3 headset to allow users to easily capture their surroundings and use it as the basis for virtual meetings. Whats more, beyond just a static capture, the Varjo Reality Cloud platform will continuously update the portion of the environment thats in the headsets view while the meeting is happening. That means if theres somethingreal and relevant in the hosts environmentlike a book, product, or even another personthe virtual viewers will be able to see that thing moving and updating in real-time (as long as the host is actively looking at it).

The idea melds well with XR-3s existing high-quality passthrough capabilities. During normal use of the headset its easy to toggle on the headsets pass-through view to see the environment around you, which means if you were sharing your local environment through the Varjo Reality Cloud, it could seem like others in the meeting were standing right in the same room as you.

Varjo thus likens its Reality Cloud platform to teleportation, though I wouldnt say it goes that far just yet.

I got to see an early prototype of the Varjo Reality Cloud in action during a meeting with the company in Silicon Valley. Using the Varjo XR-3 headset, I was shown a pre-recorded example of a Varjo Reality Cloud meeting space with a person standing in the center of the room talking, gesturing, and showing me some objects from around the room. While most of the room around me was static, the person was essentially being filmed by an XR-3 headset, which meant their movements (and anything in a certain area around them) were being updated in real-time.

To be clear, the environment I was seeing wasnt just flat or even 180 footage, it was an actual volumetric space, and so was the person that was standing inside the room. And while I could definitely make out the specific person I was looking at and the room around me, in this prototype phase the fidelity leaves a lot to be desired. The room scan and the person in front of me were assembled from a splotchy point-cloud of colored dotsfar from the incredible quality of several of Varjos photogrammetry demos that Ive seen in the past.

While its almost certain that the Varjo Reality Cloud wont look as good as careful pre-captured photogrammetry any time soon, the company says that what I was looking at is merely a proof of concept, and that improvements in fidelity are expected as they move forward with development.

One important part of that ongoing development will be moving the whole thing into the cloud. While the demo I saw was a pre-recorded example of the Varjo Reality Cloud, ultimately the company plans to stream the captured environments from the cloud to any participants in the room, leaving the bulk of the computing to be done in the cloud. To do so at the highest possible quality on its ultra-high resolution headsets, the company says it has developed a foveated compression algorithm to cut the stream down to just single megabytes per second. My understanding is that that algorithm specifically takes advantage of the eye-tracking thats built into Varjo headsets.

But Varjo headsets arent the only devices that will be able to join the Varjo Reality Cloud. While itll take an XR-3with its equipped depth-sensorsto capture and stream the environments, the company says that its taking a device-agnostic approach toparticipants. The company expects that participants joining Varjo Reality Cloud sessions could be on computers, smartphones, tablets, and other VR headsets too.

Here is the original post:

Hands-on: Varjo Reality Cloud is a Platform for Capturing & Sharing Physical Spaces in Real-time - Road to VR