Apple patent involves devices and method for providing ‘computer-generated experiences’ for the ‘RealityPro’

Apple has been granted a patent (number US 11599239 B2) for “devices, methods, and graphical user interfaces for providing computer-generated experiences.” It involves the ability of Macs, iPads, and perhaps iPhone to generated virtual reality/augmented reality scenes for viewing on the rumored “RealityPro” headset.

About the patent

The patent relates generally to a computer system with one or more display generation components and one or more input devices that provide computer-generated experiences, including but not limited to electronic devices that provide virtual reality and mixed reality experiences via one or more displays.

In the patent, Apple notes that the development of computer systems for virtual reality and augmented reality has increased significantly in recent years. Input devices, such as cameras, controllers, joysticks, touch-sensitive surfaces, and touch-screen displays for computer systems and other electronic computing devices are used to interact with virtual/augmented reality environments. 

However, Apple says that methods and interfaces for interacting with environments that include at least some virtual elements (e.g., applications, augmented reality environments, mixed reality environments, and virtual reality environments) are “cumbersome, inefficient, and limited.”

For example, systems that provide insufficient feedback for performing actions associated with virtual objects, systems that require a series of inputs to achieve a desired outcome in an augmented reality environment, and systems in which manipulation of virtual objects is “complex, tedious and error-prone, create a significant cognitive burden on a user, and detract from the user experience with the virtual/augmented reality environment,” according to the tech giant. 

In addition, these methods take longer than necessary, thereby wasting energy. Finally, many systems that provide virtual reality and/or mixed reality experiences use head-mounted display devices that physically shield the users’ faces from their surroundings, and hinder social interaction and information exchange with the outside world when the users are engaged in the virtual reality and mixed reality experiences.

Apple wants to overcome all these limitations with its RealityPro.

Summary of the patent

Here’s Apple’s abstract of the patent: “While displaying a first view of a three-dimensional environment corresponding to a first viewpoint, a computer system detects a change in grip of a user’s hand into a first predefined grip in conjunction with detecting a gaze input directed to a respective position in the three-dimensional environment. 

“If the respective position is a first position, the computer system replaces the first view with a second view of the three-dimensional environment corresponding to a second viewpoint that is selected in accordance with the respective movement of the user’s hand and the first position in the three-dimensional environment; and if the respective position is a second position, the computer system replaces the first view with a third view of the three-dimensional environment corresponding to a third viewpoint that is selected in accordance with the respective movement of the user’s hand and the second position in the three-dimensional environment.”

About the RealityPro

When it comes to the RealityPro, the rumors are abundant. Such a device will arrive this year. Or 2025, Or 2026. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be eventually be available in both. The Reality Pro may or may not have to be tethered to an iPhone to work. Other rumors say that it could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.” Or perhaps “xrOS” for extended reality operating system.