A DAB HAND AT MOCAP: THE LATEST IN FINGER TRACKING


WHAT MAKES FINGER TRACKING IN VFX TRICKY?

For visual effects and animation practitioners, gloves are the predominant method for carrying out hand and finger tracking, partly because they tend to fit into an existing ecosystem of ‘wearable’ body and facial capture hardware used in the VFX and animation workflow at a studio.

However, there are other finger tracking solutions around, too, including optical hand tracking, computer vision-based devices and even neural signal tracking devices (an example of the latter is Facebook Reality Labs’ wrist-based input device, still a research project).

Finger tracking is difficult. Think of all the dexterity in our fingers, the way we can quickly make a whole range of fast or subtle moves, the many self-occlusions that can occur as our hands and fingers cover other fingers, or as we place our hands behind our backs or in pockets or around objects. There’s also the experience of ‘drifting,’ where small changes occur to the tracked location of the fingers over time. These are the things that can make finger motion capture hard, and why several companies are out there trying to solve it for the most accurate finger tracking possible.

“What makes it an absolute extreme challenge is the fact that my hand and your hand are inherently different,” notes Bart Loosman, CEO of Manus, which makes the Prime X series of gloves, and has also partnered with Xsens to offer Xsens Gloves by Manus.

“Having a product that can then calibrate and adjust its measurements to whatever’s going on with the finger length is hard,” adds Loosman. “Every finger can bend three ways, that’s three bending points. So, you might end up having to strategically place the sensors and then just extrapolating what certain parts are doing.”

A further challenge is that innovation has been happening in body capture for many years, while finger tracking is relatively newer. “Customers, when they do hand motion capture, expect a quality on par with full-body motion capture,” comments Rob Löring, Senior Business Director for 3D body motion at Xsens. “There is still always a lot of work going on right now with finger tracking, and that’s one reason we partnered with Manus – we want to see how far we can get in perfecting our product.”

Manus’ finger tracking technology, used in its own Prime X gloves and the Xsens Gloves by Manus, rely on flex sensors and inertial measurement units (IMUs), and include the capability for haptics. Meanwhile, Rokoko, which started offering its Smartgloves in 2020, also relies on IMU sensors, similar to its Smartsuit Pro motion capture suits. CEO Jakob Balslev comments that venturing into finger tracking forced the team to rethink a lot of the logic they had used in their body tracking solvers.



Source link

More To Explore

Share on facebook
Share on twitter
Share on linkedin
Share on email