Apple wants ‘Apple Glasses’ to work with ‘synthesized reality content’

The rumored “Apple Glasses” — an augmented reality/virtual reality/mixed reality head-mounted display (HMD) — may be able to “present synthesized reality content in association with recognized objects,” as evidenced by a newly file patent (number 20200342231). 

In the patent filing, Apple notes that virtual reality (VR) and augmented reality (AR) are becoming more popular due to their “remarkable” ability to alter a user’s perception of the world. For example, VR and AR are used for learning purposes, gaming purposes, content creation purposes, social media and interaction purposes, or the like. These technologies differ in the user’s perception of his/her presence. VR transposes the user into a virtual space so their VR perception is different from his/her real-world perception. In contrast, AR takes the user’s real-world perception and adds something to it. 

Apple says that these technologies are becoming more commonplace due to, for example, miniaturization of hardware components, improvements to hardware performance, and improvements to software efficiency. As one example, a user may experience AR content superimposed on a live video feed of the user’s setting on a handheld display (e.g., an AR-enabled mobile phone or tablet with video pass-through). 

As another example, a user may experience AR content by wearing a HMD or head-mounted enclosure that still allows the user to see his/her surroundings (e.g., glasses with optical see-through). As yet another example, a user may experience VR content by using an HMD that encloses the user’s field-of-view and is tethered to a computer. 

Synthetic reality kicks things up a notch. A synthesized reality (SR) setting refers to an entirely or partly computer-created setting that individuals can sense and/or with which individuals can interact via an electronic system. In SR, an individual’s movements is monitored virtual objects in the SR setting change in a manner that conforms with one or more physical laws. 

For example, a SR system may detect an individual walking a few paces forward and adjust graphics and audio presented to the individual in a manner similar to how such scenery and sounds would change in a physical setting. An individual may interact with and/or sense a SR object using any one of his senses, including touch, smell, sight, taste, and sound.

Here’s the summary of the patent filing: “In one implementation, a method includes: obtaining image data from an image sensor; recognizing a portion of an object within the image data; obtaining synthesized reality (SR) content–such as mixed reality, augmented reality, augmented virtuality, or virtual reality content–associated with the portion of the object; and displaying the SR content in association with the portion of the object. In some implementations, the SR content is dependent on the orientation of an electronic device or the user relative to the object. In some implementations, the SR content is generated based on sensor data associated with the object.”