Content
- 1 New trends in VR device sensors
- 2 The difference between 3DoF and 6DoF
- 3 Outside-in and inside-out tracking technology
- 4 position tracking
- 5 Hand tracking revolutionizes VR headset operations
- 6 Eye tracking to reflect a person's line of sight in VR
- 7 Eye tracking enables realistic visual worlds
- 8 Dexerials products for eye tracking sensors
- 9 VR devices with more than 10 cameras and sensors
New trends in VR device sensors
Demand for AR, VR and MR headsets is expected to grow significantly with the growth of the metaverse market. Currently, games and industrial applications are the main focus. according to aStudio published by International Data Corporation (IDC)By June 2022, the virtual reality headset market is expected to grow from approximately 10 million units in 2021 to more than triple by 2026. Sensor technology is an important part of this prevailing trend. Sensor technology is not essential in the head-mounted display for watching movies or searching for videos. However, it becomes critical for AR/VR devices to play or work. This article covers recent trends in sensor technology used in AR, VR, and MR devices, as well as potential future developments.
The difference between 3DoF and 6DoF
Head-mounted displays (HMDs) project images according to the movement of the user. This is possible thanks to a system in virtual reality devices that detects the orientation and speed of rotation of the head.
Virtual reality headset systems that detect head orientation are classified as '3DoF' or '6DoF' based on the movement and direction detected by the sensor. DoF (Degrees of Freedom) refers to how much a sensor can perceive. The higher the degrees of freedom (axes) that these sensors can detect, the more immersive the VR experience will be.
3DoF means that there are three axes that can be detected: the X, Y, and Z axes. The system can recognize whether the user is looking up/down, left/right, or diagonally down/up. 3DoF compatible HMDs are mainly used for viewing 360° images. Compared to 6DoF, it has fewer features and allows users to experience virtual reality at a lower cost. Also, less space is required as you don't need to feel the movements of your hands or feet. It also allows multiple people to view VR at the same time. Because of these features, 3DoF-compliant HMDs are used in amusement parks, museums, and corporate training programs.
On the other hand, 6DoF has six freedom shafts that the sensor can detect. As shown in the illustration below, in addition to head movement, the system also detects user movement in the X, Y, and Z axis directions. Specifically, it is capable of recognizing forward/backward, left/right, and up movement. /down.
Due to the larger number of sensors, 6DoF VR headsets are more expensive than 3DoF VR headsets. It also requires a larger amount of space than is needed for 3DoF, as it is capable of detecting head and body movements. However, 6DoF VR devices can provide a more immersive user experience. The ability to provide a more realistic environment makes them suitable for realistic simulation training involving hand and body movements.
| 3DoF | 6DoF | |
|---|---|---|
| range of motion | forward/backward rotation left/right tilt left/right rotation | forward/backward rotation left/right tilt Move up, down, left, right, forward and backward |
| Merits | relatively low cost Large groups of people can use the device. | can actively move Greater immersion in VR |
| demerits | Limited use in static environment | Expensive requires more space |
3DoF is sufficient if the primary goal is to view and operate on 360° video recordings and computer graphics. However, a 6DoF-equipped device is required to enjoy a VR experience with freedom of movement (i.e., walking, crouching, and moving your hands).
Outside-in and inside-out tracking technology
The function that captures the movement of the head and neck through sensors installed in an HMD is called “head tracking”. When a user looks up, down, left or right, the sensor recognizes the movement and displays the corresponding image. This allows the user to get a 360° view within the VR CGI environment. 3DoF devices detect the orientation and direction of rotation of the head using the acceleration, gyroscope, and magnetometer sensors installed in the HMD.
On the other hand, in 6DoF devices, sensors are needed to detect the position and movement of the user's head. There are two types of systems:
- Outside In: A separate tracking camera is installed above the TV or somewhere in the room to detect head and body movements.
- Inside Out: A sensor is built into the HMD or glasses.
For the outside-in type, the cameras and sensors are located outside, so the user must be within a limited area that allows detection. On the other hand, the inside-out type uses cameras and sensors installed in the VR device to capture, measure, and analyze external images to determine the orientation of the head and the direction of movement. Therefore, there are no space restrictions and users can move freely around the room.
| outside in | Inside out | |
|---|---|---|
| Area of movement of the user | Restricted | unrestricted |
| Characteristics | Requires external cameras and sensors | No external cameras or sensors required |
position tracking
Most virtual reality headsets on the market today allow the user to control a virtual reality avatar via a hand controller. Conventional virtual reality headsets with optical sensors emit infrared rays to the outside, which are received by an external sensor to determine the user's position. This requires installing an external sensor in the room, as well as a cable to connect the sensor to the headset. On the other hand, there are now devices that don't need external sensors or cables, allowing virtual reality experiences with just a headset and controller. By making the HMD and controller 6DoF-compliant, it's possible to automatically match the user's real-world movements to their VR avatar.
These devices use the Time of Flight (ToF) method that measures the distance to an object by the time difference between light, infrared rays, ultrasonic waves, etc. emitted by the headphones or controller and the time it takes for the object to reflect the light. In recent years, this technology is being used in obstacle detection systems in autonomous vehicles.
Hand tracking revolutionizes VR headset operations
Currently, the main virtual reality devices use a hand controller to control the avatars. However, hand tracking technology is being developed as a method to enable a more immersive experience.
Hand tracking is a method in which a camera attached to an HMD recognizes the user's hands and reflects their position in the hands of the VR or MR avatar. This takes the hassle out of operating an avatar with a controller. Being able to express fingertip movements, such as waving or touching rock-paper-scissors, also enables communication through fine hand movements.
In addition, if the manual tracking function eliminates the need for a controller, the hassle of carrying and pairing controllers is also eliminated, making the VR experience more convenient. It would also eliminate hand fatigue caused by holding the controller and reduce manufacturing costs. In the future, users will be able to intuitively control VR spaces, such as pausing and fast-forwarding a video through hand gestures.
Hand tracking uses various cameras and artificial intelligence technologies to accurately determine the position and movement of your fingers. In addition, a technology that reproduces the sensation of touch through a glove controller (haptic technology) is being developed. There is potential for various technological developments related to hands on virtual reality.
Eye tracking to mirror a person's line of sight in virtual reality
Along with the use of hands in VR operations, eye-tracking sensors that use the movements of the human eye are also advancing. Eye-tracking sensors are installed inside a virtual reality headset to capture the user's eye movements and reflect the user's line of sight in virtual reality.
One of the drawbacks of the VR experience is "VR motion sickness", similar to motion sickness. However, eye tracking can help alleviate this problem. By mirroring eye movements in virtual space, users can look at other avatars, make eye contact, or aim at a target in a shooter using only their line of sight. Today's eye-tracking sensors use a near-infrared camera to capture light reflection points on the cornea and eyeball to estimate the direction and position of gaze.
Eye tracking enables realistic visual worlds
When a person focuses on one thing, the surrounding area becomes blurry. However, with common VR headsets, all projected images are clearly in focus. This causes a phenomenon in which the viewer feels a difference with reality, reducing their feeling of immersion. The eye tracking feature is effective in removing this uncomfortable feeling.
New developments in eye tracking will also enable a technology called "forbidden rendering," which uses a near-infrared camera to detect the user's eye movements and project high-resolution images only near the field of view.
Rendering only the areas the user is focused on at high resolution, as well as rendering at a lower resolution as you zoom out, can significantly reduce the computational load on the device and greatly improve overall graphics quality. Eye tracking technology is expected to enable visual expressions that are closer to real life, leading to enhanced VR experiences with higher visual quality.
Dexeriales Products for Eye Tracking Sensors
Dexeria's Anisotropic Conductive Film (ACF) and Anti-Reflective Film (ARF) contribute to the development and quality improvement of eye-tracking sensors.
Eye tracking sensors are usually placed around the eye of the device. When sensors use film substrates, they are generally optically transparent. FPCs (flexible printed circuit boards) are sometimes glued onto optically clear films for wiring infrared LED signs, and an increasing number of customers are turning to Dexeriais ACFs for this purpose. Eye tracking sensor modules can be made thinner, smaller and lighter when joined with ACF. In addition, Dexeriales anti-glare and anti-glare films can be used to suppress light reflection inside the case.
VR devices with more than 10 cameras and sensors
In the future, there will be VR and MR devices that use various sensors and camera modules to combine outdoor images with VR images. These devices can be equipped with 10 or more cameras and sensor modules. ACF and Dexeriales' thermosetting and UV adhesives used to bond the lenses could contribute to the development of VR and MR devices.
Furthermore, through the improvement of various tracking technologies, future VR and MR devices will be able to accurately recognize fine movements of the head, neck, body, gaze, hands, etc. of a person, generating images, sounds and tactile sensations. Dexerials will continue to contribute to the development of new devices for the metaverse.