Interactive Kiosk Software: New Ways to go TouchFree

Posted; September 22, 2021

Adding gesture control to self-serve kiosks and interactive digital signage just got a lot more exciting. Last year, we launched TouchFree – an application for retrofitting touchscreens with gesture control. But that was just the beginning. Our new TouchFree Tooling enables developers to weave touchless interaction more tightly into user interfaces. We can’t wait to see what new user experiences this unlocks.

By Dean Kendall, Product Manager

We launched TouchFree in late 2020. It’s an application that runs invisibly on top of existing touchscreen interfaces, providing a touchless cursor that users control using mid-air hand movements.  

TouchFree won the Self-Service Innovation Award for Best COVID-19 Solution. But while the pandemic catalyzed demand for touchless kiosks, the appeal of touchless technology goes far beyond immediate hygiene concerns.

Touchless technology is converging with ultra-fast 5G connectivity and IoT. Together, they’re powering new user experiences across self-serve kiosks, interactive digital signage, and experiential marketing.

And we know from talking to designers and developers that they want to be able to use TouchFree to do more than simple retrofits.

That’s why in addition to the existing TouchFree Application, we’re now releasing TouchFree Tooling for Web and Unity. With it, you can move beyond retrofits and design interfaces with a gesture-first approach.

We’ve also developed a new Touch Plane interaction that enables quicker, more intuitive user journeys. Particularly suitable for transactional self-serve kiosks, it’s available for both TouchFree Tooling and the retrofit TouchFree Application.

TouchFree 1.0 took Ultraleap hand tracking data and used it to position a cursor on a 2D screen using a transparent overlay.

TouchFree 2.0 still does this. But in addition, new TouchFree Tooling allows Web and Unity developers to connect applications and content to TouchFree and use its hand positioning data directly.

TouchFree Tooling integrates quickly and seamlessly into your codebase. It gets a touchless cursor into Web or Unity kiosk applications in a matter of minutes.

A better way to think about it is what can’t you do?!

TouchFree Tooling gives you complete control over cursor design and how your application reacts to users’ hand movements. We envisage it being used to develop truly innovative self-serve kiosks and interactive digital signage user experiences.

To get you started, here are some things we’ve been working on:

This release also includes a new interaction style, Touch Plane.

When we say “interaction style”, what we mean is what hand movements the user performs to interact with the screen.

Ultraleap’s core hand tracking software digitally models the position of users’ hands with a high degree of accuracy. It works at distances of up to a metre away from the camera module.

Theoretically, a touchless kiosk application could allow users to interact using almost any hand movement. However, in practice we’ve found users accustomed to touchscreens naturally gravitate towards certain gestures. TouchFree 1.0 included options for the Air Push and Hover & Hold interaction styles.

Based on our ongoing research, we’ve now added a new Touch Plane interaction. This creates an interaction plane, very much like a virtual touchscreen hovering a few centimetres in front of the physical screen.

Touch Plane retains all the benefits of Ultraleap’s tracking accuracy, but also makes it straightforward to complete simple, transactional interactions with maximum speed.

The new, expanded TouchFree empowers designers and developers to seamlessly integrate gesture control into self-serve kiosks. Or go even further, and push the boundaries of user experience in interactive digital signage and experiential marketing.

We’re sure there are many applications of TouchFree we haven’t even imagined yet! Share the projects you’re developing with us at @ultraleap or @ultraleap_devs.

Dean Kendall is the Product Manager for out-of-home software solutions at Ultraleap.