Apple has files for a patent (number 20190258058) for a “head-mounted display for virtual reality with inside-out positional, user body and environmental tracking.” It’s one of many patents the company has filed for — or already been granted — for the rumored “Apple Glasses,” an augmented reality/virtual reality headset.
Per the patent info, the head-mounted display (HMD) would use computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency would be achieved by performing part of the processing on the HMD itself.
In the patent filing, Apple notes that the objective of immersion in a virtual world is to convince a user’s mind to perceive a non-physical world as if it were real. The concept of reality here refers more to the notion of perceptual plausibility rather than representing a real world.
In virtual reality (VR), immersion is achieved by displaying computer generated graphics that simulate a visual experience of a real or imagined world. The quality of immersion is subject to several important factors. For instance, characteristics of the display such as image quality, frame rate, pixel resolution, high dynamic range (HDR), persistence and screen-door effect (i.e., the visible lines between pixels on the screen).
Apple says the quality of the immerse experience decreases when the displayed field of view is too narrow or if the various tracking functions are slow and/or inaccurate (leading to disorientation and nausea; otherwise known as simulation sickness). Immersion is also impacted by the camera system performance such as the image quality (noise, dynamic range, resolution, absence of artifacts) and the coherence between the virtual graphics (3D modeling, textures and lighting) and the pass-through images.
In mixed reality (MR), virtual elements are composited in real-time into the real world environment seen by the user. Physical interaction between the virtual elements and real world surfaces and objects can be simulated and displayed in real-time.
Tracking of various elements is generally recognized as an essential prerequisite for achieving a high end VR and MR application experience. Among these elements, positional head tracking, user body tracking and environment tracking play a key role in achieving great immersion. Apple wants to make sure the “Apple Glasses’ accomplish all these goals whenever it debuts. That will be, per most forecasts, 2020 or 2021.
