Apple patent involves ways to improve Personas on the Apple Vision Pro

Apple has been granted a patent for “Apple patent involves ‘Online Modeling for Real-Time Facial Animation.” It apparently involves improving the quality of Personas on the Apple Vision Pro.

About the patent

Vision Pro users can create a digital Persona that allows others to see them while they’re wearing the headset. Apple says it’s “a dynamic, natural representation of your face and hand movements while you’re using FaceTime. However, the feature has been frequently criticized for offering “creepy” Personas. Apple wants to change this.

The patent relates to a method for real-time facial animation and, in particular, to a processing device and a real-time facial animation system. Moreover, the disclosure relates to a dynamic expression model which may include a plurality of blendshapes that may be used to track facial expressions of a user in order to generate a corresponding graphical representation.

Summary of the patent

Here’s Apple’s abstract of the patent: “Embodiments relate to a method for real-time facial animation, and a processing device for real-time facial animation. The method includes providing a dynamic expression model, receiving tracking data corresponding to a facial expression of a user, estimating tracking parameters based on the dynamic expression model and the tracking data, and refining the dynamic expression model based on the tracking data and estimated tracking parameters. The method may further include generating a graphical representation corresponding to the facial expression of the user based on the tracking parameters. Embodiments pertain to a real-time facial animation system.”