Apple has been granted a patent (number 11,165,989) for “gesture and prominence in video conferencing” that involves improving FaceTime conference sessions.
About the patent
The patent involves a multipoint audio conference, live audio input at conference source endpoint devices are shared as a conference with other receiving endpoints devices. Apple says that, typically, each receiving device can hear the audio from every source device by rendering a mixture of audio from multiple source devices.
In some cases, a human participant listing to the mixed audio may differentiate between simultaneous speakers as they would when people speak over each other during an in-person meeting. In other cases, an audio-conferencing system may improve clarify of communication by giving auditory prominence to one or a subset of streams from source devices, instead of mixing audio from all source devices equally. Apple says that auditory prominence may include attenuating the volume or muting the non-prominent audio sources. For example, a squelch mechanism may mute audio sources with lower audio input levels.
A multipoint video conference allows multiple live video feeds to be shared amongst conference participants. At each receiving device, video feeds of participants at captured at remote locations are typically presented in separate windows of a single display. A video conference may also include audio, which may be shared as in an audio conference, including squelching of non-prominent speakers.
Summary of the patent
Here’s Apple’s abstract of the patent: “Techniques are presented for managing for visual prominence of participants in a video conference, including conferences where participants communicate visually, such as with sign language. According to these techniques, a visual prominence indication of a participant in a video conference may be estimated, a video stream of the participant may be encoded, and the encoded video stream may be transmitted along with an indication of the estimated visual prominence to a receiving device in the video conference.”
