Dev Tool Uses Quest Hand-tracking to Quickly Model Realistic Hand Poses for VR

Grabbing objects in VR may be one of the medium’s most fundamental interactions, but making it work well isn’t as easy as you might think. Developers often need to spend time hand animating the hand model so that it appears to hold each object in a realistic way. Developer Luca Mefisto has built a smart tool which uses Quest’s hand-tracking to enable developers to motion-capture hand-poses, making the whole ordeal quicker, more realistic, and ultimately more immersive for players.

Update (July 20th, 2020): Developer Luca Mefisto has released the first version of his HandPosing tool which uses Oculus Quest hand-tracking to quickly author realistic hand-poses for virtual reality interactions. The tool is available on GitHub.

“This is a work in progress, and things are subject to change. I hope it serves others either as a useful tool or at least as a starting point for their grabbing-interaction implementations,” Mefisto writes.

The original article below, which outlines the benefits and functions of the HandPosing tool, continues below.

Original Article (July 7th, 2020): Some VR games employ various methods of ‘dynamic’ animation to create realistic hand poses when players grab objects in VR (Lone Echo, for instance). Generally that’s proprietary tech, which means any developers wanting to do the same would need to build a similar system from scratch (not an easy task).

Rather than do that, some games cut out the hand-posing problem entirely by simply making your virtual hands disappear when you grab objects (Vacation Simulator, for instance).

Developers that want to keep the player’s hands visible need to create hand-poses manually so that when you grab an object, your virtual hand grips the object in a realistic way. It’s not that this is a difficult task per se, but it can be immensely time consuming.

At minimum you need one custom hand-pose for every uniquely shaped object in a given game. Even then, consider how many different ways players might want to hold a single object… even if you cut out unlikely poses, you still may need four or five poses for a single object to cover the most obvious grips. If there’s 100 uniquely shaped objects in a game, that could mean animating 400 or 500 hand-poses.

VR developer Luca Mefisto wants to make this whole process quicker and easier—allowing developers to make more realistic poses in less time. He’s building a tool which smartly leverages Quest’s hand-tracking feature to allow developers to take a ‘snapshot’ of their own of their own hand gripped around virtual objects.

The tool then allows developers to define valid positions for the pose, allowing the hand to snap realistically to the nearest valid position on the object.

Objects can also have multiple poses and grabbing points to cover different ways of grabbing the same object (like the scissors below).

Though the tool requires Quest’s hand-tracking for creating the poses, Mefisto says the hand-pose tool will work for games that employ hand-tracking or controllers.

Though the tool is so far unnamed, the developer plans to release it as an open source project to the VR development community. You can follow Mefisto on Twitter to see updates on the tool’s development and keep an eye out for its release.