Apple has filed for a patent (number 20210174567) for a “method for facial animation.” The goal is to use 2D images, 3D depth maps and animation priors, to achieve tracking of the facial expressions of a user.
The patent filing could offer new ways to create realistic Animoji and Memoji features. Or it could involve ways to create content on Macs and/or iPads.
Animojis allow a user to choose an avatar (e.g., a puppet) to represent themselves. The Animoji can move and talk as if it were a video of the user. Animojis enable users to create personalized versions of emojis in a fun and creative way. Memoji is the name used for Apple’s personalized “Animoji” characters that can be created and customized right within Messages by choosing from a set of inclusive and diverse characteristics to form a unique personality.
In the patent filing, the company notes that capturing and processing human geometry, appearance, and motion is at the core of modern computer animation. Digital actors are often created through a combination of 3D scanning and appearance acquisition. Motion capture is used to animate the digital actors.
The problem, per Apple, is that these existing methods of creating and animating digital actors require “complex acquisition systems and substantial manual post-processing.” As a result, creating and animating digital actors using existing methods, requires long turn-around times and substantial production costs.
What’s more, Apple says that existing method for animating digital actors lack “robustness.” In other words, the existing methods fail in the presence of noise sources — such as when the inputs to the animation system are corrupted by noise. Apple wants to overcome these advantages.
