Apple has been granted a patent (number 11,043,018) for a “video pipeline” that involves the rumored “Apple Glasses.” That’s the anticipated augmented reality/virtual reality/mixed reality head-mounted display (HMD).
Virtual reality (VR) systems display virtual views that provide an immersive virtual environment. Mixed reality (MR) systems combine virtual content with a view of the real world, or add virtual representations of real world objects to a virtual environment.
Conventional VR and MR systems are typically either tethered systems including a base station that performs at least some of the rendering of content for display and a device connected to the base station via a physical connection. Alternately, stand-alone devices may perform the rendering of content locally.
Apple says that stand-alone systems allow users freedom of movement. However, because of restraints including size, weight, batteries, and heat, stand-alone devices are generally limited in terms of computing power. This means they’re limited in the quality of content that can be rendered. The base stations of tethered systems may provide more computing power and thus higher quality rendering than stand-alone devices. However, the physical cable tethers the device to the base station and thus constrains the movements of the user.
Apple’s solution: a mixed reality system that includes a device and a base station that communicate via a wireless connection.
Here’s the summary of the patent: “The device may include sensors that collect information about the user’s environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection. The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display.
“The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.”
When it comes to Apple Glasses, such a device will arrive this year or 2022, depending on which rumor you believe. The Sellers Research Group (that’s me) thinks Apple will at least preview it before the end of the year.
It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be eventually be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”
