Apple has filed for a patent (number 20210286502) for “devices, methods, and graphics user interfaces for providing computer-generated experiences.” It involves interaction with virtual reality (VR) and augmented reality (AR) on devices such as the Mac with, for instance, hand gestures. It also involves the rumored “Apple Glasses,” a VR/AR head-mounted display (HMD).
About the patent filing
In the patent filing, Apple notes that the development of computing systems for AR has increased significantly in recent years. Examples of AR environments include at least some virtual elements that replace or augment representations of the physical world.
Input devices, such as cameras, controllers, joysticks, touch-sensitive surfaces, and touch-screen displays for computing systems and other electronic computing devices are used to interact with virtual/augmented reality environments. Examples of virtual elements include virtual three-dimensional objects, digital images, video, text, icons, and control elements such as buttons and other graphics.
However, Apple says that methods and interfaces for interacting with environments that include at least some virtual elements (e.g., applications, AR environments, mixed reality environments, and VR environments) are “cumbersome, inefficient, and limited.”
The tech giant adds that, for example, “systems that provide insufficient feedback for performing actions associated with virtual objects, systems that require a series of inputs to achieve a desired outcome in an augmented reality environment, and systems in which manipulation of virtual objects are complex, tedious and error-prone, create a significant cognitive burden on a user, and detract from the experience with the virtual/augmented reality environment.”
In addition, Apple says that these methods take longer than necessary, thereby wasting energy. This latter consideration is particularly important in battery-operated devices.
In addition, many systems that provide VR and/or mixed reality experiences use HMDs that physically shield the users’ faces from their surroundings, and hinder social interaction and information exchange with the outside world when the users are engaged in the virtual reality and mixed reality experiences. Apple wants to overcome such limitations.
Summary of the patent filing
Here’s Apple’s abstract of the patent filing: “A computing system displays, via a first display generation component, a first computer-generated environment and concurrently displays, via a second display generation component: a visual representation of a portion of a user of the computing system who is in a position to view the first computer-generated environment via the first display generation component, and one or more graphical elements that provide a visual indication of content in the first computer-generated environment.
“The computing system changes the visual representation of the portion of the user to represent changes in an appearance of the user over a respective period of time and changes the one or more graphical elements to represent changes in the first computer-generated environment over the respective period of time.”
About Apple Glasses
When it comes to Apple Glasses, such a device will arrive in 2022 or 2023, depending on which rumor you believe. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be eventually be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a cU.S.tom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”
Figure 4 illustrates a hand tracking unit of a computing system that is configured to capture gesture inputs of the user.
