Apple wants you to be able to even more quickly edit faces and objects in pics you’ve snapped with your iPhone or iPad. The company has been granted a patent (number 11,282,543) for “real-time face and object manipulation.”
About the patent
In the patent filing Apple notes that the combination of short focal length and small distance between an object and a camera can create a projection of the object in images captured by devices such as iPhones and iPads. Such iamges may seem distorted, especially in comparison to images of the object captured with longer focal lengths and longer distances to the camera.
The short focal length of a wide-angle lens may create a type of lens distortion called barrel distortion. The short distance to the center of an object relative to the differences in distance to different points on an object may seem to distort the projection as portions of the object closer to the camera appear larger than portions of the object further from the camera.
For example, a selfie photo or video is a self-portrait of the person holding the camera. The object in a selfie image is typically the face of the person who is holding the camera in their hand at arm’s length, or less, from their face.
Cameras on the front face of a smartphone typically have a shorter focal length than the higher-quality cameras on the back face of a smartphone. This combination of a short focal length of a front-face camera with short arm’s length distance between camera and face produce images with a projection that may appear distorted, particularly in comparison to non-selfie images of the same face taken at a longer distance by the back-face camera on the same smartphone.
For example, with a nose of a face centered and close to a camera, the nose will appear large relative to the ears of the same face because the nose will be closer to the camera. Also, because barrel distortion will enlarge items in the center of the image while shrinking items toward the edges of an image.
Apple wants iPhone and iPad users to be able to quickly fix such distortions on the photos they take.
Summary of the patent
Here’s Apple’s abstract of the patent with technical details: “Techniques are presented for modifying images of an object in video, for example to correct for lens distortion, or to beautify a face. These techniques include extracting and validating features of an object from a source video frame, tracking those features over time, estimating a pose of the object, modifying a 3D model of the object based on the features, and rendering a modified video frame based on the modified 3D model and modified intrinsic and extrinsic matrices. These techniques may be applied in real-time to an object in a sequence of video frames.”
