Apple has filed for a patent (number 20210312662) for “generating body pose information.” It involves the rumored “Apple Glasses,” an augmented reality/virtual reality head-mounted display (HMD).
About the patent filing
In the patent filing, Apple notes that some devices are capable of presenting computer-generated reality (CGR) experiences. For example, some HMDs present immersive CGR experiences to a user of the HMD. Some CGR experiences require knowing a body pose of the user.
For example, some CGR experiences present an avatar of the user that mimics the behavior of the user. If the user moves a portion of his/her body, the avatar moves the corresponding portion. In such CGR experiences, presenting accurate avatars requires knowing a body pose of the user.
In some CGR experiences, the CGR experience is altered based on the body pose of the user. For example, as the user moves, the scene being presented in the CGR experience changes. In such CGR experiences, providing a realistic CGR experience requires knowing a body pose of the user. Apple wants its Apple Glasses to be able to do just that.
Summary of the patent filing
Here’s Apple’s abstract of the patent filing: “A method includes obtaining a set of images that correspond to a person. The method includes generating a body pose model of the person defined by a branched plurality of neural network systems. Each neural network system models a respective portion of the person between a first body-joint and a second body-joint as dependent on an adjacent portion of the person sharing the first body-joint.
“Providing the set of images of the respective portion to a first one and a second one of the neural network systems. The first one and second one correspond to adjacent body portions. The method includes determining, jointly by at least the first one and second one of the plurality of neural network systems pose information for the first respective body-joint and the second respective body-joint.”
About Apple Glasses
When it comes to Apple Glasses, such a device will arrive in 2022 or 2023, depending on which rumor you believe. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be eventually be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”
