Apple has filed for a patent (number 10,586,372) for “online modeling for real-time facial animation” involving its Animoji and Memoji features, as well as other applications.
Animojis allow a user can select an avatar to represent themselves. The Animoji can move and talk as if it were a video of the user.
Animojis enable users to create personalized versions of emojis in a fun and creative way. Memoji is the name used for iOS’s personalized Animoji characters that can be created and customized right within Messages by choosing from a set of inclusive and diverse characteristics to form a unique personality.
In the patent filing, Apple says that recent advances in real-time performance capture have brought within reach a new form of human communication. Capturing dynamic facial expressions of users and re-targeting these facial expressions on digital characters enables a communication using virtual avatars with live feedback. However, a successful deployment of facial animation technology at a large scale puts high demands on performance and usability.
Apple notes, however, that state of the art marker-based systems, multi-camera capture devices, or intrusive scanners commonly used in high-end animation productions are not suitable for consumer-level applications. The company wants to change this.
Here’s the summary of the invention: “Embodiments relate to a method for real-time facial animation, and a processing device for real-time facial animation. The method includes providing a dynamic expression model, receiving tracking data corresponding to a facial expression of a user, estimating tracking parameters based on the dynamic expression model and the tracking data, and refining the dynamic expression model based on the tracking data and estimated tracking parameters. The method may further include generating a graphical representation corresponding to the facial expression of the user based on the tracking parameters. Embodiments pertain to a real-time facial animation system.”
