10 Reasons the Best VR Controller is No Controller

Posted; November 27, 2019

Today’s sensor and hand tracking technology lets us leave handheld controllers behind and interact with VR using only our hands. Here are the top 10 reasons why free hands are often the best VR controllers of all.

While VR interaction has become more standardized in the past few years, there’s still a significant learning curve with every new experience. With buttons, analogue sticks, and triggers, it takes time to figure out what does what. How about reaching out and interacting naturally instead?

If you look beyond home gaming and into enterprise applications, you find a very different kind of user – one who is less accustomed to console controllers, and might not even be able to use them.

For these users, the learning curve with VR controllers feels more like a mountain – and they often don’t have the time or inclination to climb it. The struggle with tools designed for dedicated gamers is driving everyone from industrial engineers to eye clinics to product designers towards hand tracking as the universal interface.

As humans, we have many ways to interact and communicate with each other using our hands. Body language is at least as important as what we actually say. Waving, pointing at something, or high fiving in VR feels a lot more natural than “press X for thumbs up.”

It’s hard to beat mid-air interactions for hygiene. A study by UNICEF and Domestos suggested home gaming controllers carry five times more bacteria than toilet seats.

This isn’t a big problem in your own home (though you might want to wipe that controller down occasionally). But for shared VR controllers, it’s a serious concern. Controllers used in public settings such as retail installations, out-of-home entertainment venues, and medical VR are as frequently handled as public touchscreens – and as likely to be contaminated with faecal bacteria.

Never worry again about fumbling through a junk drawer for a replacement battery. Your hands require no accelerometers, gyroscopes, positional tracking LEDs, linear oscillators, or Bluetooth chips.

Hand tracking technology is small, simple, and can be hidden inside your VR headset. The most important hardware is your hands, and they come standard.

When it comes to mobile VR, handheld controllers also require extra effort and take up more space. It’s the difference between throwing one piece of kit into your travel bag, or three.

One of the most powerful ways to create presence in VR is simply to be able to look down and see your hands, reacting and moving in real time. Seeing this tells your brain it’s real.

Fumbling around for VR controllers after you put the headset on is also pretty distracting and bad for immersion. And in large-scale VR, such as theme park rides and art installations where each user’s time is limited, it wastes valuable minutes.

Today’s controllers (both VR and standard consoles) come with sophisticated haptic feedback as standard. We wouldn’t want to lose that, right? Luckily, we don’t have to. Contactless haptics – such as Ultraleap’s own ultrasonic technology – now make it possible to experience the sense of touch in mid-air.

Goodbye, hardware engineering. Hello, evolution. With 27 degrees of freedom and the ability to detect an object the size of a molecule, your hands are awesomely acute and powerful tools honed by millions of years of development.

There’s a reason hand tracking is one of the greatest challenges in 21st-century computer science – the human hand is extraordinarily complex. No one knows this better than us. We’ve gone through ten years and five generations of software development to get to our latest Gemini platform.

Today’s hand tracking is far more robust than previous generations of software. It enables us to bring the full power of your hands into VR and unlock richer and more nuanced interactions.

Hands can do everything a VR controller can, but they can also do more.

For many applications, simple binary actions (pinch, grab, push, trigger) are all you need. But if you want users to be able to sculpt digital materials or manipulate sound waves as though they were physical objects, that’s possible too. Hand tracking lets you develop ways of interacting that fit the application ­– not the other way around.

What happens in AR doesn’t stay in AR. No one will want to walk down the street or through the office with controllers strapped to their wrists, making hand tracking the only viable interface for AR. As VR and AR blend together under the broader categories of XR and spatial computing, a single universal interface will need to bridge these various platforms and user experiences.

Handheld controllers will always be popular accessories for home VR, as long as people want to wield guns or swords. But with the rise of mobile platforms, AR, and applications beyond home gaming in enterprise, theme parks, industrial design, and healthcare, the future of VR interaction in the 2020s is literally in your hands.