Editors Note: Valve has started shipping the prototypeKnuckles controllersto select VR developers in limited quantities. UploadVR hasnt gotten its hands on a pair just yet, so weve reached out to the development community to see what they think. This article is a contribution from Denny Unger, Co-Founder, CEO, and Creative Director of Cloudhead Games, the studio behind The Gallery: Episode 1 The Call of the Starseed and Episode 2 Heart of the Emberstone.

User interfaces in virtual reality start with your hands. We know that now with the Oculus Touch and the HTC Vive, but even when VR was simply a screen strapped to your head many felt that hands were the future. We began to develop The Gallery on the first Oculus devkit with the Razer Hydra (a Sixense technology) to deliver surrogate hand tracking and body presence back in 2013. And when Valve invited Cloudhead and the first wave of developers to see what would eventually be known as the Vive, Valve showed a commitment to that same vision. Now Valve has invited us and a new wave of developers to begin working with their latest prototypethe SteamVR Knuckles, a wearable VR controller that tracks not just your hand, but each finger too.

Back in 2014, we didnt realize exactly how accurate SteamVR tracking wasthe whole notion of roomscale VR was almost incomprehensible. Up until that point, there was still some motion latency in VR, so you never felt completely attached to the actions in your hands. But once the SteamVR Lighthouses started tracking objects in a 3D space, it was a deep and immediate connection of, Holy crap, thats actually my hand in VR.

Our goal then was to deliver an experience where the player doesnt have to think about the controller, and has only natural, gestural interactions. We wanted to demonstrate why this kind of inputyour hands themselveswas meaningful. When we received our first Vive devkit (wired at the time) we were taping them to our hands in order to feel more immersed, and we even spoke to Valve about crude ways they could strap the controller on.

Those early Vive prototypes already showed an incredible level of fidelity, capable of measuring the tiniest fraction of a movement. Its like the Moores Law of motion control; each incremental improvement in tracking brings with it new possibilities. So as soon as you have that kind of fidelity with your hands in VR, you need your fingers to be more purely represented. And thats what the Oculus Touch started to do by bringing capacitive, gestural input to the controller.

Where Touch differs from what the Knuckles offer, however, is that youre still pushing a binary button in the end; Touch feels more grounded in traditional gamepad design. Specifically, you always feel like youre holding something with buttonsand that works perfectly for gun games and sword games. But the Knuckles take that further by removing the abstractions of first-gen VR inputs. Even though it looks like a more complicated device, its actually a much simpler one.

With the Knuckles, youre not holding a controller; it simply straps to your hand and rests in your palm. If you relax your hand into a natural flex, the controller stays put and keeps tracking your fingers. If you reach out to grab an object in VR, your hand wraps around the base of the controller, giving a tangible feeling of grabbing something. That physicality is something you dont get from data gloves, or vision based inputs without any device, and that feeling can then be fine-tuned with haptic feedback. Plus, youre not passing around a sweaty data glove between your friends.

When reaching out to an object with the Knuckles, Im not thinking about the controller Im holding in my hand, because Im not holding one. Im not thinking about how to use my fingers either, because theyre not assigned to a button press. Im not even thinking about my hand, and thats where the magic comes inIm just thinking about grabbing the object, as I would in real life. That entire grasping motion is represented in VR, whether I pinch with two fingers, scoop with my hand, or close my fist around it. The Knuckles track your fingers by the distance they are from the base of the controller (your palm), and represent that movement in VR. Its second nature.

As developers, when we receive prototype hardware like the SteamVR Knuckles, it makes us want to push the capabilities. In the past, thats meant radically rethinking our stack of interactions and locomotion systemsvirtually redesigning the game. Weve written about what these kinds of changes have meant for The Gallery in the past, but the long story short is that new controllers like the Knuckles arent just affecting the complexity of interactions. We now have new possibilities for game mechanics and design that havent and couldnt have been done before. Its to be seen how these controllers will impact Heart of the Emberstone in September, but theyll be a core focus in designing Episode 3.

Think of an interface-heavy app like Tilt Brush. Dials can be intuitive, but using your fingers is organic. Theres a possibility for gestural movements to call functions and navigate dense data; there could be an entire language built out of using your hands to manipulate paint brushes and pencils and sizes and colors. Once you take the mental load of an interface off the playeronce they stop thinking about the controlleryou can leverage that partition into experiential design and organic controls.

The kind of technology that the SteamVR Knuckles offer is not just impactful to the future of input in gaming, but also the future of output. I can go to a social VR space and point to something, or offer a peace sign, or tell somebody to hang loose without having to think about it. The controller doesnt guess your gesture, or snap to a new position, it represents your fingers based on the distance it calculates. The more natural and intuitive the interface, the less we think about hardware. And the less we fixate on hardware, the more present we can be in VR.

Prior to the Knuckles, hardware developers were looking for something that would be more broadly accepted by the general public. Something that resembled a Wiimote, like the Vive wands, or something that when put together resembled a gamepad, like the Touch controllers, meant that VR input was familiar. Strapping an alien device to your hand in first-gen VR would have been too much too fast. But I think it took the evolution of those two controllers to get VR to the point where the public could be comfortable with the idea of a controller strapped to your hand.

There are so many moments in life in which using your hands is a vital part of the experience. There are implications for education and communicationwith audiences who dont generally understand videogame controllersbecause the SteamVR Knuckles open the door to that broader audience. These are pick-up-and-play controllers where you dont have to think about the input, you just reach out and interact in virtual space.

User interface in virtual reality starts with your hands. And once users are empowered in that way, and dont have to be told how to use the technology, the next generation of virtual reality is here.

This is a guest post not produced by the UploadVR staff. Its a contribution submitted by Denny Unger, theCo-Founder, CEO, and Creative Director of Cloudhead Games.No compensation was exchanged for the creation of this content.

Tagged with: Knuckles, prototype, valve

See the original post:

Hands-On With Valve's Knuckles Prototype Controllers - UploadVR