Apple has been granted a patent (10,417,824) for a “method and system for representing a virtual object in a view of a real environment.” It applies to the company’s augmented reality work on iOS devices and could apply to the rumored “Apple Glasses,” an augmented readily/virtual reality headset expected to debut in 2020 or 2021.
Apple’s goal with the invention is to illuminate or display virtual objects with the same lighting conditions visible in the real world, as well as let the virtual objects change the illumination for example by casting shadows onto parts from the real scene. For many augmented reality scenes, lighting conditions are typically unknown and arbitrary. Apple says it’s generally difficult or even impossible to have consistent lighting, but the company wants to change this.
Here’s the summary of the invention: “The invention relates to a method of representing a virtual object in a view of a real environment which comprises the steps of providing image information of a first image of at least part of a human face captured by a first camera, providing at least one human face specific characteristic, determining at least part of an image area of the face in the first image as a face region of the first image, determining at least one first light falling on the face according to the face region of the first image and the at least one human face specific characteristic, and blending in the virtual object on a display device in the view of the real environment according to the at least one first light. The invention also relates to a system for representing a virtual object in a view of a real environment.”
Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.
