Apple patent involves ‘object detection using multiple three dimensional scans’

Apple has been granted a patent (number 11,100,659) for “object detection using multiple three dimensional scans.” It would seem to involve the iPhone, iPad, and the rumored Apple Glasses and an ability for them to scan objects in 3D.

About the patent

In the patent, Apple notes that various electronic devices include image sensors that capture images of the real world environment. For example, many mobile devices include image sensors that can be used to capture a sequence of frames (e.g., video frames) that are presented on the displays of such devices or stored for later viewing. 

Apple says that detecting and tracking the objects that appear in such frames is desirable for a variety of applications. The tech giant says that such detection and tracking can be facilitated by capturing images of an object and then using those images to detect the object in subsequent images. 

However, Apple notes that objects may not be detected and tracked when the capture conditions do not match the detection conditions. For example, if the lighting when the capture is performed is different than the lighting when the detection is performed, the object may not be detected. 

Summary of the patent

Here’s Apple’s abstract of the patent: “One exemplary implementation facilitates object detection using multiple scans of an object in different conditions. For example, a first scan of the object can be created by capturing images of the object by moving an image sensor on a first path in a first condition, e.g., bright lighting. “A second scan of the object can then be created by capturing additional images of the object by moving the image sensor on a second path in a second condition, e.g., dim lighting. Implementations determine a transform that associates the scan data from these multiple scans to one another and use the transforms to generate a 3D model of the object in a single coordinate system. Augmented content can be positioned relative to that object in the single coordinate system and thus will be displayed in the appropriate location regardless of the condition in which the physical object is later detected.”

The accompanying graphic illustrates the path of an image sensor.