Apple is going all in on augmented reality (AR). The company has filed for a patent (number 20170301141) for a “method of image processing for an augmented reality application.” It involves ARKit, which allows developers to tap into the latest computer vision technologies to build detailed virtual content on top of real-world scenes for interactive gaming, immersive shopping experiences, industrial design and more.
In the patent filing, Apple notes that AR systems and apps present enhanced information of a real environment by providing a visualization of overlaying digital information, particularly computer-generated virtual information, with visual impressions or an image of the real environment. The digital information can be any type of visually perceivable data such as objects, texts, drawings, videos, or any combination thereof.
The real environment is captured, for example, with a camera held by a user or attached on a device held by a user. The digital information has to be superimposed with the real environment or a part of the real environment in the camera image at a right time, at a right place and in a right way in order to offer a satisfied visual perception to users. Therefore, Apple says it would be beneficial to provide a method of image processing in an augmented reality application which is capable to improve the performance and usability of augmented reality applications, particularly augmented reality authoring tools.
The right time requires that the digital information should be superimposed with the real environment in the image when and only when it is required or necessary, for example, when a particular real object appears in the camera image.
Here’s Apple’s summary of the invention: “An apparatus for and method of image processing in an augmented reality application is provided. The method includes the steps of: providing at least one image of a real environment; performing image processing in an augmented reality application with the at least one image employing visualization of overlaying digital information with visual impressions or the image of the real environment and employing vision-based processing or tracking; and adjusting at least one of a parameter and operating flow of the vision-based processing or tracking depending on at least one of the following: a usage of the image processing, a usage of the visualization, a visually perceivable property of the digital information or the real environment, a property of a display device employed in the visualization, or a manner in which a user is viewing the visualization.”
Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.
