Hand tracking technology and haptic feedback explained
To create a more immersive and interactive experience, virtual reality (VR) and mixed reality (MR) headsets use various types of technology, including hand tracking and haptic feedback.
These features mix digital and physical to enhance user experience and enjoyment. They can also significantly improve business outcomes, from increasing memorability and differentiation to boosting productivity and creativity through VR/MR.
Let’s take a look at the benefits of hand tracking technology and VR/MR haptic feedback for your teams.
Introduction to hand tracking technology in MR
Hand tracking in VR/MR gives people more freedom – it means they don’t need to use a controller while in the metaverse. They can choose to interact with objects using natural hand movements, or switch between their hands and a controller. VR/MR hand gestures can also make an avatar’s movements look more realistic to others.
Hand tracking technology works by using the inside-out cameras on Meta Quest headsets. These cameras detect and track hand position, orientation and finger configuration. With Direct Touch on Meta Quest, people can directly tap and scroll on panels with their fingers. They can swipe through content, tap on objects and move panels as if on a physical tablet. Many hand tracking movements are based on the familiar ‘pinch’ gesture you may have seen people use in VR or MR.
People can disable hand tracking on their VR/MR controller at any time by toggling it off in Settings > Movement tracking > Hand and body tracking. Once disabled, Meta Quest uses the position and orientation of the headset and controllers instead.
What is haptic feedback in MR?
Haptic feedback in VR/MR is designed to recreate the perception of touch in the metaverse. Haptics connect what a user sees in the metaverse to what they feel in their body.
Vibrations or ‘rumbles’ are an example of haptic feedback already used in a wide range of video games. Vibrations also provide haptic feedback when a user ‘taps’ something, commonly on a smartphone or smartwatch. Furthermore, haptics are used in VR/MR apps to make the experience more interactive and tactile – for example, by simulating the feeling of resistance when pushing a virtual door. More sophisticated uses, such as recreating the feeling of rainfall, can make the VR/MR experience even more memorable.
Meta Quest Touch Pro and Meta Quest Touch Plus controllers use TruTouch haptics for more realistic feedback. Sensors in the triggers, thumb rest and grip detect subtle movements, such as the user bending or sliding their index finger. The aim of TruTouch haptics is to feel as though the controllers are natural extensions of your hands.
Advantages of hand tracking and haptic feedback for business
There are many benefits to MR hand tracking and haptic feedback technology for businesses:
Intuitive training: These technologies create a sense of immersion and provide a learning experience that’s more focused on behavior than theory. A person trained in a virtual simulation using haptics and hand tracking is more likely to know what to do in a real-life situation. 60% of survey respondents agreed VR/MR simulations gave them confidence in applying skills in real life – rising to 82% within the energy and utilities sector.1
Improved accessibility: Hand tracking may be more accessible than touchscreens or keyboards for some employees – especially those who are less able to use controllers or have cognitive challenges.
Unlocking creativity: Working in a new way can help to unlock creativity within organizations, boosting innovation and ROI. For example, 76% of survey respondents agreed that VR/MR helps to increase the pace of innovation in their organizations.2
Increased productivity: A more hands-on approach can improve product design, product testing and speed of iterations. Global snack brand Mondelēz used VR/MR to reduce its 3D concept design phase from weeks to hours.
Differentiation: Hand tracking and haptic technology could help businesses stand out to prospective employees and consumers, especially if used in marketing and recruitment initiatives.
Product testing: Hand tracking can be used for virtual product testing, where haptic feedback helps consumers connect with and evaluate a product in a VR/MR setting.
Hand tracking and haptic feedback in action
Hand tracking technology and haptic feedback have helped the following organizations:
Pfizer used hand speed tracking and haptic feedback to train staff on keeping a medical environment sterile. When an operator used their hands incorrectly within the VR/MR simulation, for example, the Meta Quest headset produced vibrations as an alert.
Vehicles for Change created a VR/MR training simulation program for mechanics where a virtual instructor led students through a series of 15 key tasks. Each task involved replicating the hands-on intricate nature of mechanical training – without a real garage. Hand movements needed to be precise to ensure pinpoint accuracy.
The Centre for Healthcare Innovation (CHI) in Singapore used VR/MR to create a lifelike simulation of a blood-taking room to train nursing staff. By including visual, audio and tactile engagement of the senses, the team provided an enriching VR/MR training experience that was positively rated by 100% of participants.
Get in touch with Meta for Work
Discover how Meta Horizon managed solutions can scale your organization with virtual and mixed reality. Learn how you can use VR and MR to build the future of work and education with our success stories. Getting started is easy, please
Keep reading about working in mixed reality (MR):
What’s the best VR headset for work?
Why spatial audio is the sound of the future
Sources
- “Survey: VR at Work, 2023” by CCS Insight (Meta-commissioned study of 500 UK and US respondents using VR at work), Feb 2024.
- Insight “Survey: VR at Work, 2023” by CCS Insight
