Apple has applied for two patents regarding augmented reality (AR), both of which would logically (or so it seems to me) involve the iPhone. One patent (number 9,560,273) is for a “wearable information system” having at least one camera.
The system will have at least one camera and a low-power mode and a high power mode. The high-power mode would be activated by a detection of at least one object in at least one field of view of the at least one camera.
In the patent filing, Apple says that AR, as a new user interface paradigm, “has seen great progress, especially based on computer vision algorithms for object recognition and pose estimation. “Head mounted displays, equipped with a camera have been known for some time.
Apple says that one big obstacle for the success of AR-based information systems, which are able to always watch the user’s surrounding for interesting objects, is the high power-consumption of image processing algorithms run on the application processor or the GPU (graphics processing unit). The company is proposing a wearable information system which has a camera that may take a low-resolution image and match it against a known symbol (e.g. the face of the user of the phone or an image) in low power mode.
If, in the low-resolution image a known symbol is found, the wearable information system (which Apple also describes as a phone) may wake up and take a higher resolution image to verify the result and basically unlock the phone in order to take a call or similar things. The goal is to enable power-efficient unlocking, based on images. A scenario might be the phone is laying around and as soon as it receives a call, it tries to check if the user gets in sight in order to unlock the screen.
Patent number 9,558,581 is for a “method for representing virtual information in areal environment.” It provides steps of providing a system setup with at least one display device, wherein the system setup is adapted for blending in virtual information on the display device. The invention also relates to a computer program product comprising software code sections for performing the method.
In the patent filing, Apple notes that one major function of AR systems is to overlay items of virtual information, such as points of interest (POIs), to a view of the real environment. This is particularly useful and popular in location-based (mobile) augmented reality applications, such as tour guides. For example, users could use an augmented reality application to overlay POIs to a view of the real environment when they are in new places and want to obtain information about things they see.
Apple notes, that in augmented realty systems, the POI information has to be represented in the real environment or the part of the real environment such that it satisfies desired visual perception and usability. Most data sources for POI information provide the information in relation to a point in the real world, not as a 3D model with more than one point or vertex. Apple says its invention “is well suited for handling the visualization of items of virtual information, which are described by latitude, longitude and optionally altitude and additional meta-information consisting of text and 2D image information.”
In January the Financial Times reported that Apple has “assembled a large team of experts in virtual and augmented reality and built prototypes of headsets…”.
The article (behind a paywall) notes that the research team has “hundreds of staff” that have been assembled from acquisitions and poaching employees from companies in the space such as Microsoft and Lytro. One of the acquisitions is Flyby Media, a company that worked with Google on a 3D positioning technology. Apparently the team has also been building VR/AR headset prototypes for several months.
