Apple has been granted four new patents for its rumored “Apple Glasses,” an augmented reality/mixed reality (AR/MR) head-mounted display (HMD).
Patent number 10,747,312 is for “image enhancement devices with gaze tracking.” This would enable Apple Glasses to tell where a user was looking and adjust the AR/MR view, accordingly.
Here’s how it would work: Control circuitry in the device can gather information on a user’s point of gaze using a gaze tracking system and other sensors, can gather information on the real-world image such as information on content, motion, and other image attributes by analyzing the real-world image, can gather user vision information such as user acuity, contrast sensitivity, field of view, and geometrical distortions, can gather user input such as user preferences and user mode selection commands, and can gather other input.
Based on the point-of-gaze information and/or other gathered information, the control circuitry can display the real-world image and supplemental information on the display. The supplemental information can include augmentations such as icons, text labels, and other computer-generated text and graphics overlaid on the real world image and can include enhanced image content such as magnified portions of the real-world image.”
Patent number 10,748,969 is for a “display with holographic angle-of-view adjustment structures.” Its purpose is to adjust the viewing angle of a user wearing a HMD.
Here’s how it would work: the display would have an array of pixels. Each pixel may have a light-emitting diode such as an organic light-emitting diode or may be formed from other pixel structures such as liquid crystal display pixel structures. The pixels would emit light such as red, green, and blue light.
An angle-of-view adjustment layer may overlap the array of pixels. During operation, light from the pixels passes through the angle-of-view adjustment layer to a user. The viewing angle for the user is enhanced as the angular spread of the emitted light from the pixels is enhanced by the angle-of-view adjustment layer.
Patent number 10,748,340 is for an “electronic device with coordinated camera and display operation” that could be configured to display virtual reality (VR) content for a user in which no real-world content from the camera is displayed or mixed reality content in which a combination of real-world content from the camera and overlaid virtual reality content is displayed. The goal is to have seamless video with no distracting visual artifacts.
Control circuitry in the device would adjust the display and camera while transitioning between VR and MR modes. The control circuitry may reconfigure the camera to exhibit a desired frame rate immediately upon transitioning from VR mode to MR mode. Transitions between modes may be accompanied by smooth transitions between frame rates to avoid visible artifacts on the display.
Patent number 10,748,331 is for “3D lighting” and involves techniques for displaying a graphical element in a manner that simulates three-dimensional (3D) visibility. In the patent filing, Apple notes that the realistic display of 3D objects on a 2D surface has been a long-time goal in the image processing field. One approach to simulating a 3D object is to take a large number of images each illuminated from a different position.
There are various methods to doing this. Apple’s approach is to take a number of images, each captured with a known spatial relationship to a target 3D object, and use them to construct a lighting model of the target object.
When it comes to Apple Glasses, such a device will arrive next year or 2022, depending on which rumor you believe. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”
