Embodied Labs: Hand Tracking in Healthcare VR

Posted; March 9, 2021

Embodied Labs’ award-winning VR experiences give healthcare workers a first-person look inside what it’s like to live with common conditions that affect seniors – such as Alzheimer’s and macular degeneration. Using hand tracking, rather than VR controllers, is essential to ensuring accessibility for users who have often never experienced VR before.  

Embodied Labs are pioneers in the use of virtual reality in healthcare training. Their immersive experiences provide users with a unique, first-person look inside what it’s like to live with common conditions that affect seniors. From the start, using hand tracking rather than VR controllers has been an integral feature.

Founder Carrie Shaw set up Embodied Labs following her own mother’s diagnosis with early onset Alzheimer’s. Each experience is an engaging narrative designed to replicate real, authentic experiences as closely as possible.

Learners embody people such as Beatriz, living with Alzheimer’s Disease; Alfred, who has macular degeneration; or Eden, a transgender older adult who embodies common experiences transgender older adults have had that involve decades of discrimination, stigma, negativity, and trauma. The experiences give learners an understanding and empathy they can’t get from traditional training tools.

The Frank Lab was released in late 2020 and is their most sophisticated VR experience yet. Learners embody 72-year-old Frank. Beginning a few months after his wife’s passing, the experience is in two modules. In the first, the learner experiences the destructive impacts of social isolation through Frank’s eyes.

In the second module, learners embody Frank in exactly the same storyline. This time, however, they experience how, with the proper support, Frank can have better health, connect with his family and community, and find more purpose and meaning in life.

The Frank Lab has a branching storyline to increase replay value. Powered by Ultraleap’s hand tracking, it also has more interactive objects than any previous Embodied Labs VR experience.

Embodied Labs’ end users (for example, senior living workers) have often never experienced VR before. How do you ensure learners can access these powerful experiences easily and intuitively?

This isn’t a trivial problem. Someone who’s just spent hundreds of dollars on a consumer headset may well be happy to do a 15-minute onboarding experience. But VR training use-cases are different. Every minute spent learning button pushes is time not spent doing the actual learning people signed up to do.

Controllers also risk putting off users less familiar and comfortable with new technologies. For these users, the learning curve for VR controllers can feel like a mountain – and can put them off the entire training experience.

From the start, Embodied Labs solved this problem by using Ultraleap’s hand tracking in preference to VR controllers. Being able to see their hands, reacting and moving in real time, also increases users’ sense of embodiment.

Users can pick up a brain, then travel inside it to learn about how it functions at the neuron level in a healthy brain, versus one with Alzheimer's disease. They can reach out to receive a pair of hearing aids from their doctor to enable them to hear fully again, or swat away lightning bugs that have appeared as a visual hallucination due to Lewy Body Dementia.

Typically, Embodied Labs use our VR Developer Mount to attach a Leap Motion Controller to the front of a VR headset. In the near future, as VR headsets start to feature Ultraleap hand tracking as a native feature, integration will be even easier.

The introduction of touchscreens made 2D computing far more accessible. Users – such as the very young or very old – who would struggle or be completely unable to use keyboard and mouse input can now access content with ease.

Embodied Labs’ pioneering healthcare application is an example of how hand tracking does the same for VR – and why this matters. By opening VR up to wider demographics and increasing accessibility, Embodied Labs are able to use technology as a powerful force for good.

Their immersive training platform is used in healthcare across America by senior living organizations such as Front Porch, Kendal Corporation, and Benedictine Living. It helps caretakers empathize with and better understand the conditions most commonly experienced by residents. Organizations who work with the families of older adults living with chronic conditions also use it to help family members.

In education, colleges such as The University of New England and Penn State College of Medicine use Embodied Labs' VR experiences to teach medical students. Even consumer companies use it to train customer service staff who interact frequently with older adults. 

Embodied Labs have won multiple awards and featured in publications such as the New York Times, Voices of America, and CNBC. In 2020, they raised $3.2 million of seed funding, led by top age-tech investor, Ziegler Link·Age Fund.

We’re proud to be part of their journey.