Apple granted a patent for ‘emojicon puppeting;

At some point you may be able to use “puppeting” to create new emojis for Apple’s Memojis and Animojis. The tech giant has been granted a patent (number 201803367166714) for “emojicon puppeting.”

An emoji is a small graphic or icon that can convey an expression or an emotion, such as a smiley face that conveys happiness or a kiss that conveys affection. A user of a messaging system may include an emoji in a message to express the emotion conveyed by the emoji. An emoji may have a fixed animation, such as the emoji winking an eye, or blowing a kiss. However, a user can’t alter the fixed animation of the emoji, and Apple wants to change this. 

In the patent, the tech giant notes that some animation and augmented reality (AR) systems can add objects to an image. For example, an AR system may add a funny nose to a picture of a person, or morph a picture of one person, such as the user, into a picture of another person, such as a famous actor or public figure. 

These systems typically use camera output (e.g. RGB data) to determine contours of the person to provide the data to perform the morphing. Using only RGB data for depth information can be problematic in low-light conditions. Apple thinks it has a solution.

Here’s the summary of the invention: “systems and methods for generating a video of an emoji that has been puppeted using inputs from image, depth, and audio. The inputs can capture facial expressions of a user, eye, eyebrow, mouth, and head movements. A pose, held by the user, can be detected that can be used to generate supplemental animation. 

“The emoji can further be animated using physical properties associated with the emoji and captured movements. An emoji of a dog can have its ears move in response to an up-and-down movement, or a shaking of the head. The video can be sent in a message to one or more recipients. A sending device can render the puppeted video in accordance with hardware and software capabilities of a recipient’s computer device.”