‘Apple Glasses’ could allow you to directly manipulate virtual objects in CGR environments

Apple has filed for a patent (number 20200241629) for “manipulation of virtual objects using a tracked physical object.” It involves the rumored “Apple Glasses” and would allow users to move a physical object to manipulate an onscreen augmented reality/virtual reality/mixed reality (AR/VR/MR) image.

In the patent data, Apple notes that various electronic devices exist, such as head-mound devices (also known as headsets and HMDs), with displays that present users with a computer-generated reality (CGR) environment in which they may be fully immersed in a surrounding physical environment, fully immersed in a virtual reality environment comprising virtual objects, or anywhere in between. While direct manipulation of physical objects in the surrounding physical environment is available to users naturally, the same is not true for virtual objects in the CGR environment.

Apple says that lacking a means to directly interact with virtual objects presented to a user as part of a CGR environment limits a degree to which the virtual objects are integrated into the CGR environment. The tech giant wants to provide users with a means of directly manipulating virtual objects presented as part of CGR environments. 

Here’s Apple’s summary of the invention: “In one implementation, a method of manipulating virtual objects using tracked physical objects is disclosed. The method involves presenting content including a virtual object and a virtual representation of a proxy device physically unassociated with an electronic device on a display of the electronic device. 

“Input is received from the proxy device using an input device of the proxy device that represents a request to create a fixed alignment between the virtual object and the virtual representation in a three-dimensional (“3-D”) coordinate space defined for the content. The fixed alignment is created in response to receiving the input. A position and an orientation of the virtual object in the 3-D coordinate space is dynamically updated using position data that defines movement of the proxy device in the physical environment.”

When it comes to Apple Glasses, such a device will arrive next year or 2022, depending on which rumor you believe. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”