Macs may one day be able to generate 3D images from pics taken with an iPhone

Macs may one day be able to generate 3D images from pics and videos taken with an iPhone, iPad, or with the FaceTime camera on an Apple desktop or laptop. The tech giant has applied for a patent (number 20200125835) for “generating animated 3D models from captured images.”

Motion capture has been used in a variety of areas to generate motion data that is based on tracking and recording the movements of real objects. For example, motion capture technology has been used frequently in video game production and movie production.

However, Apple says that motion capture technology, however, has not been widely implemented at the consumer level. Consumer level motion capture systems have just begun to be implemented as processing and power advancements begin to allow consumer based electronics to more readily perform operations associated with motion capture. Apple wants to change this.

Here’s the summary of the patent filing: “A three-dimensional model (e.g., motion capture model) of a user is generated from captured images or captured video of the user. A machine learning network may track poses and expressions of the user to generate and refine the three-dimensional model. Refinement of the three-dimensional model may provide more accurate tracking of the user’s face. 

“Refining of the three-dimensional model may include refining the determinations of poses and expressions at defined locations (e.g., eye corners and/or nose) in the three-dimensional model. The refining may occur in an iterative process. Tracking of the three-dimensional model over time (e.g., during video capture) may be used to generate an animated three-dimensional model (e.g., an animated puppet) of the user that simulates the user’s poses and expressions.”