Apple has filed for a patent (number 20190188915) for a “method and apparatus for representing a virtual object in a real environment.” It involves the company’s continued work in augmented reality/virtual reality (AR/VR), as well as its almost-certainly-coming “Apple Glasses” head-mounted display (HMD).
The invention relates to a method and apparatus for representing a virtual object in a real environment, particularly for merging the virtual object with an image of the real environment generated by a recording device. This is one of many patents filed for and granted to Apple. “Apple Glasses” is the name used to refer to the augmented reality/virtual reality headset the company is expected to debut next year or 2021
Here’s the (very wordy)summary of the patent filing: “The invention relates to a method for representing a virtual object in a real environment, having the following steps: generating a two-dimensional representation of a real environment by means of a recording device, ascertaining a position of the recording device relative to at least one component of the real environment, segmenting at least one area of the real environment in the two-dimensional image on the basis of non-manually generated 3D information for identifying at least one segment of the real environment in distinction to a remaining part of the real environment while supplying corresponding segmentation data, and merging the two-dimensional image of the real environment with the virtual object or, by means of an optical, semitransparent element directly with reality with consideration of the segmentation data. The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.”
Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.
