Apple wants iPhones, iPads to capture full 3D sound in video recordings

Apple wants to improve the audio of videos make with your iPhone and iPad. The company has filed for a patent (number 20200137489) for “spatially biased sound pickup for binaural video recording.”

In the patent filing, Apple notes that binaural recording of audio facilitates a means for full 3D sound capture–in other words, being able to reproduce the exact sound scene and giving the user a sensation of “being there.” This can be accomplished through spatial rendering of audio inputs using head related transfer functions (HRTF), which modifies a sound signal in order to induce the perception in a listener that the sound signal is originating from any point in space. 

However, Apple says that while this approach is compelling for, for example, full virtual reality applications, in which a user can interact both visually and audibly in a virtual environment, in traditional video capture applications three dimensional sounds can distract the viewer from the screen. In contrast, monophonic or traditional stereophonic recordings may not provide a sufficient sense of immersion. Apple wants to change this

Here’s the (rather technical) summary of the patent filing: “A method for producing a target directivity function that includes a set of spatially biased HRTFs. A set of left ear and right ear head related transfer functions (HRTFs) are selected. The left ear and right ear head HRTFs are multiplied with an on-camera emphasis function (OCE), to produce the spatially biased HRTFs. The OCE may be designed to shape the sound profile of the HRTFs to provide emphasis in a desired location or direction that is a function of the specific orientation of the device as it is being used to make a video recording.”