Apple granted patent for a head-mounted projector (say what?)

Apple has been granted a patent (number US 11652965 B2) for a “method of and system for projecting digital information on a real object in a real environment.” It involves a projection system mounted on a user’s head. No, really.

About the patent

In the patent Apple says that augmented reality (AR) systems could enhance a real environment by directly visually augmenting the real environment by computer-generated digital information. For example, such digital information is virtual information for augmenting visual impressions of the real environment. 

Typical applications are known as, for example, so-called projector-based AR, projective AR or spatial AR, such as referred to in reference . The digital information can be any type of visually perceivable data such as objects, texts, drawings, videos, or their combination. The direct augmentation of the real environment could be realized by projecting the computer-generated digital information onto a surface of a real object of the real environment or a part of the real environment using a projector.

Apple apparently feels that one such implementation of the patent is a head-mounted projection system.

Summary of the patent

Here’s Apple’s abstract of the patent: “A method of projecting digital information on a real object in a real environment includes the steps of projecting digital information on a real object or part of a real object with a visible light projector, capturing at least one image of the real object with the projected digital information using a camera, providing a depth sensor registered with the camera, the depth sensor capturing depth data of the real object or part of the real object, and calculating a spatial transformation between the visible light projector and the real object based on the at least one image and the depth data. The invention is also concerned with a corresponding system.”