VR Flight Training: Lufthansa Aviation Training & NMY

Posted; February 17, 2022

Lufthansa Aviation Training’s VR cabin crew program was the first VR training program approved by the German Aviation Authority (Luftfahrt Bundesamt). Developed by NMY Mixed Reality Studio, since its introduction in 2019 around 20,000 Lufthansa flight attendants have used it to complete a part of their annual training. Hand tracking was essential to this innovative and effective immersive learning program. 

In-person cabin crew training programs are enormously expensive in both time and money. Courses require physical dummy aircraft, or operational aircraft being temporarily grounded to allow training to take place.

Lufthansa Aviation Training wanted to develop a VR training program that could substitute part of this. Their focus initially was on the mandatory refresher training for security searches. These are searches flight attendants perform after every flight. They ensure no suspicious or potentially dangerous items have been left on board.

Lufthansa’s brief to NMY Mixed Reality Studio was very clear. This enterprise use of VR could only be successful with hand tracking, instead of gaming controllers.

NMY selected Ultraleap hand tracking because of its reliability and flexibility. Unlike some other hand tracking solutions, Ultraleap hand tracking isn’t tied to any specific headset range.

Ultraleap hand tracking cameras range from headsets with fully integrated tracking through to peripherals. This meant NMY could easily integrate hand tracking with their preferred HTC Vive headset.

Participants must go through the VR flight training in just 30 minutes. NMY rapidly discovered that users had no time to learn complex gestures or intricate interactions.

What NMY realized, however, was that this didn’t matter. Counter-intuitively, while a broad similarity to the way flight attendants interact naturally in a real aircraft was essential, exact fidelity was unimportant.

A flight attendant may need to pick up a cushion to search under it. However, they can pick up the cushion any way they like. What they are learning is not how to pick up a cushion, but the correct process to search an aircraft effectively and deal with a potentially dangerous object.

With this in mind, NMY implemented two important innovations behind the scenes:

In practice this means that attendants can “pick up” a cushion by simply putting their hand in the general area. The cushion sticks to their hand. They can move quickly through the experience and focus on the order of the security search – not on learning VR interactions ultimately irrelevant to the purpose of the training.

NMY also used a simple virtual wearable on the user’s left wrist. The virtual watch tells them how much time they have left. When they turn over their hand, they are provided with an itemized list of tasks to complete.

With real lives at stake, the German Aviation Authority is rightly cautious when it comes to approving innovative training programs. This makes the fact that Lufthansa Aviation’s training was the first VR flight training program to have been approved a huge achievement and a big step forward in immersive training.

Since its introduction in 2019 around 20,000 Lufthansa flight attendants have used the VR program to complete a part of their annual training. VR flight training clearly demonstrated its efficiency and cost-effectiveness in a mainstream, highly safety-conscious industry. None of this would have been possible without hand tracking.