Apple has filed for a patent (number 20220210562) for a “system to move sound into and out of a listener’s head using a virtual acoustic system.”
About the patent filing
The human auditory system modifies incoming sounds by filtering them depending on the location of the sound relative to the listener. The modified sound involves a set of spatial cues used by the brain to detect the position of a sound. Human hearing is binaural — in other words, it uses two ears to perceive two sound-pressure signals created by a sound.
Sound is transmitted in air by fluctuations in air pressure created by the sound source. The fluctuations in air pressure propagate from the sound source to the ears of a listener as pressure waves.
The sound pressure waves interact with the environment of the path between the sound source and the ears of the listener. In particular, the sound pressure waves interact with the head and the ear structure of the listener. These interactions modify the amplitude and the phase spectrum of a sound dependent on the frequency of the sound and the direction and the distance of the sound source.
In the patent filing, Apple notes that these modifications can be described as a Head Related Transfer Function (HRTF) and a Head-Related Impulse Response (HRIR) for each ear. The HRTF is a frequency response function of the ear. It describes how an acoustic signal is filtered by the reflection properties of the head, shoulders and most notably the pinna before the sound reaches the ear.
The HRIR is a time response function of the ear. It describes how an acoustic signal is delayed and attenuated in reaching the ear, by the distance to the sound source and the shadowing of the sound source by the listener’s head.
Apple is interested in a virtual acoustic system — an audio system (e.g., a digital audio signal processor that renders a sound program into speaker driver signals that are to drive a number of speakers). This would give a listener the illusion that a sound is emanating from somewhere in space when in fact the sound is emanating from loudspeakers placed elsewhere.
One common form of a virtual acoustic system is one that uses a combination of headphones (e.g., earbuds) and binaural digital filters to recreate the sound as it would have arrived at the ears if there were a real source placed somewhere in space. In another example of a virtual acoustic system, crosstalk cancelled loudspeakers (or cross talk cancelled loudspeaker driver signals) are used to deliver a distinct sound-pressure signal to each ear of the listener.
Binaural synthesis transforms a sound source that does not include audible information about position of the sound source to a binaural virtual sound source that includes audible information about a position of the sound source relative to the listener. Binaural synthesis may use binaural filters to transform the sound source to the binaural virtual sound sources for each ear. The binaural filters are responsive to the distance and direction from the listener to the sound source.
Apple’s plans could involve future versions of the AirPods, AirPods Pro, AirPods Max, HomePod mini, or a return of the larger, original HomePod.
Summary of the patent filing
Here’s Apple’s abstract of the patent filing: “In a device or method for rendering a sound program for headphones, a location is received for placing the sound program with respect to first and second ear pieces. If the location is between the first ear piece and the second ear piece, the sound program is filtered to produce low-frequency and high-frequency portions.
“The high-frequency portion is panned according to the location to produce first and second high-frequency signals. The low-frequency portion and the first high-frequency signal are combined to produce a first headphone driver signal to drive the first ear piece. A second headphone driver signal is similarly produced. The sound program may be a stereo sound program. The device or method may also provide for a location that is between the first ear piece and a near-field boundary.”
