Apple invention would facilitate an AR navigation system, virtual interior design planning, more

Apple has applied for a patent (number 20170109929) for a “method and device for illustrating a virtual object in a real environment” that would allow the representation of a digital object in a real environment. The tech giant says this would be useful for augmented reality (AR) applications such as in vehicles, in interior design planning, factory planning, prototype construction or test construction in product development. 

One possible use of the invention is the change of a real room, for example in order to place virtual furnishings in said room for which free space is required. An advantage of augmented reality technology is the viewing of objects in their real environment — for example, the viewing of a wooden table on an existing wooden floor or in front of an existing cabinet. You could virtually remove spurious real furniture pieces from a scene or to change their position in the room without having to clear them aside physically. 

The invention can not only be employed for room planning, however, but also for other augmented reality (AR) applications, for example an AR navigation system in a vehicle. Perhaps in an upcoming version of CarPlay.

Apple’s invention would generate a two-dimensional image of a real environment by means of a recording device (an iPhone, perhaps), ascertaining a position of the recording device relative to at least one component of the real environment. It would segment at least one area of the real environment unmarked in reality in the two-dimensional image for identifying at least one segment of the real environment in distinction to a remaining part of the real environment. 

It would do so while supplying corresponding segmentation data, and merging the virtual object with the two-dimensional image of the real environment with “consideration of the segmentation data such that at least one part of the segment of the real environment is removed from the image of the real environment.” The invention permits any collisions of virtual objects with real objects that occur upon merging with a real environment to be represented in a way largely close to reality.

Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.