Apple has been granted a patent (number 10,565,796) for a “method and system for compositing an augmented reality scene” that shows it has big plans for augmented reality (AR) content and interaction on its various devices, such as the rumored “Apple Glasses.”
Such a device will arrive this year, next year, or 2021, depending on which rumor your believe. It may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.
The newly granted patent also involves Apple’s ARKit. It allows developers to tap into the latest computer vision technologies to build detailed virtual content on top of real-world scenes for interactive gaming, immersive shopping experiences, industrial design and more.
In the patent data, Apple notes that virtual content (such as AR content) is usually stored within a database (library) or model file, and is fetched by the AR system for rendering. In other words, the content is digitally stored, and any interaction with it, such as editing, requires using dedicated computer software.
Content libraries/databases in AR systems are commonly maintained either locally on the augmenting device, or remotely on a database server. Updating local libraries on user devices is a relatively complicated task (especially if the devices are distributed between a large number of users). Updating remote libraries (databases) is a relatively simple and centralized task, yet, as Apple notes, it requires connecting to a data network, such as the Internet or a cellular network
The tech giant says there’s a need for a AR method and system, wherein AR content is embedded within an image (e.g., a sketch) by eliminating the need for downloading the AR content from a local or remote library/database, and allowing interaction with the the embedded content by hand sketching on the image. Also, there’s is a need for the content to employ dual perception. Dual perception means that the the embedded AR content is visually understandable also without using any AR system. Finally, Apple says there’s is a need to provide a way to easily interact with AR content.
Here’s the summary of the patent: “Disclosed are systems and methods for compositing an augmented reality scene, the methods including the steps of extracting, by an extraction component into a memory of a data-processing machine, at least one object from a real-world image detected by a sensing device; geometrically reconstructing at least one virtual model from at least one object; and compositing AR content from at least one virtual model in order to augment the AR content on the real-world image, thereby creating AR scene.
“Preferably, the method further includes; extracting at least one annotation from the real-world image into the memory of the data-processing machine for modifying at least one virtual model according to at least one annotation. Preferably, the method further includes: interacting with AR scene by modifying AR content based on modification of at least one object and/or at least one annotation in the real-world image.”
