Why Gemini Matters: 5 Examples from VR in Training

Posted; October 26, 2021

Ultraleap Gemini is our fifth-generation hand tracking platform. The most robust, flexible hand tracking ever, it works even in challenging situations – such as when one hand partly obscures the other. In this blog, we’ll get granular on the benefits of Gemini by exploring it through the lens of a specific use case – VR in training.

By Matt Tullis, Director, Spatial Computing at Ultraleap

It’s the first day of a VR training course for a new crop of flight attendants, and the atmosphere’s both excited and apprehensive.  

Statistically, it’s easier to get into an Ivy League college than become a flight attendant. This is a dream come true for the trainees sat in the room. They’re desperate to succeed. And hanging on the wall, just over there, is a row of VR headsets. 

Gemini is the most advanced hand tracking ever. But to really grasp what this means, and why it matters, you need to look at VR from the perspective of end users such as those trainees.  

Some of the trainees feel confident. Maybe they even own their own consumer VR headset. But to others – such as the 76% of Americans who have never tried VR – while the technology is exciting, it’s also strange and a little bit scary. 

Some have never even had much experience of game controllers. Or perhaps they have – and not liked them.  

But where do you start with designing a hand tracking experience?

The trainees know they’re being continuously assessed. What if they can’t figure out how to use VR?  

Observing from the back of the room is the airline’s training and development manager. She knows VR could save her department hundreds of thousands of dollars. But she’s also battling inertia and scepticism within the organisation – and her own mind. 

Can interacting in VR really match interacting with full-scale dummy aircraft? Will everyone be able to use VR training successfully, and how will they feel about it? Last but not least, can it be effectively scaled?  

Just using hand tracking instead of VR controllers makes a big difference. But how good the hand tracking you’re using is fundamentally impacts success too. 

Here’s five specific examples of how Ultraleap Gemini – the best hand tracking ever – helps our trainee flight attendants learn effectively in VR.   

Gemini has lightning-fast initialization. Add the wide field of view of our Stereo IR 170 camera, and your hands will already be tracked before they come into view. 

Your hands are your primary means of interacting with the world. They’re an essential part of your sense of presence – which is what makes immersive learning successful in the first place.  

With Gemini, there’s no disorienting moment when trainees wonder “Where are my hands?”. They can see them immediately, and start interacting with little or no instruction. It reduces cognitive load, streamlines onboarding, and deepens immersion and confidence from the very first second. 

Gemini’s also very good at adapting to different sizes and shapes of hands – so the 5’ 3” flight attendant trainee in the front row and the 6’ 1” trainee in the back will both be able to use VR equally. 

We evolved to have two hands for a reason. For VR training to match the muscle memory built by training with dummy aircraft, it requires not just being able to use your hands, but use them together.  Open a tray table. Push a trolley.  

Gemini makes it easier than ever before to use two hands in VR. Your hands can be close together or even partly obscuring each other, without hand tracking dropping out or other unexpected behaviour.   

With Gemini, VR training powered by hand tracking can scale beyond controlled training environments and into back-room warehouses, offices, or even trainees’ homes. 

Gemini’s advanced algorithms together with infrared cameras make hand tracking robust in a wide variety of lighting conditions. It can also pick out hands even against cluttered backgrounds. 

Unlike some other hand tracking solutions, Gemini isn’t tied to any specific headset range. There’s a range of hardware options, from headsets with fully integrated hand tracking such as Varjo or Lynx, through to using our Leap Motion Controller and VR Developer Mount as a peripheral. 

Gemini has also been rearchitected from the ground up to deliver equally high quality hand tracking on both mobile devices and tethered PC VR systems. 

Flight attendant training in VR doesn’t require ultra-high-fidelity visuals. Our training manager can run Gemini on affordable and accessible mobile headsets powered by the Qualcomm XR2 chip. But Gemini will also work on the high-end PC VR flight simulators for pilot training, over on the other side of the building. 

While this blog dives into the specific example of VR in training, the features we’ve called out are also applicable to other use cases. 

Being able to interact naturally with two hands matters for ergonomic and design reviews. Accommodating different sizes and shapes of hands matters for VR arcade experiences. And fast, reliable initialization matters for projects such as Embodied Labs’ VR empathy training.  

Gemini makes your virtual hands mirror your real ones more closely than ever before. And this reliable, real-time virtual embodiment of your hands creates VR that’s accessible to everyone, everywhere.  

Matt Tullis is Director, Spatial Computing at Ultraleap. He has almost 20 years of experience in the consumer electronics industry and has worked in the XR ecosystem at both Tobii and Immersion.