Future Animojis and Memoji may add voice effects based on facial expressions

In the future Apple’s Memojis and Animojis may team up with Siri and Messages to “speak” in tones that match the facial expression of the avatar characters. The tech giant has filed for a patent (number 20180336716) for “voice effects based on facial expressions.”

In the patent filing, Multimedia content, such as emoji’s, can be sent as part of messaging communications. Apple  notes that emojis can represent a variety of predefined people, objects, actions, and/or other things, and that some messaging applications allow users to select from a predefined library of emojis that can be sent as part of a message that can contain other content (e.g., other multimedia and/or textual content). 

Animojis are one type of this other multimedia content, where a user can select an avatar (e.g., a puppet) to represent themselves. The animoji can move and talk as if it were a video of the user. Animojis enable users to create personalized versions of emojis in a fun and creative way, and Apple thinks equipping them with appropriate vocal inflections would add to the fun.

Here’s Apple’s summary of the invention: “Embodiments of the present disclosure can provide systems, methods, and computer-readable medium for adjusting audio and/or video information of a video clip based at least in part on facial feature and/or voice feature characteristics extracted from hardware components. For example, in response to detecting a request to generate an avatar video clip of a virtual avatar, a video signal associated with a face in a field of view of a camera and an audio signal may be captured. 

“Voice feature characteristics and facial feature characteristics may be extracted from the audio signal and the video signal, respectively. In some examples, in response to detecting a request to preview the avatar video clip, an adjusted audio signal may be generated based at least in part on the facial feature characteristics and the voice feature characteristics, and a preview of the video clip of the virtual avatar using the adjusted audio signal may be displayed.”

Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.