Apple has filed for a patent (number 20200058152) for a “video pipeline” for a mixed reality (MR) system that involves a “base station.”
It could involve a Mac serving as the base station and communicating wirelessly with the anticipated “Apple Glasses.” The AR/VR/MR will arrive this year, next year, or 2021, depending on which rumor you believe.
In the patent filing, Apple notes that vrtual reality (VR) and augmented reality (AR) systems display virtual views that provide an immersive virtual environment. Mixed reality (MR) systems combine virtual content with a view of the real world, or add virtual representations of real world objects to a virtual environment. Conventional VR and MR systems are typically either tethered systems including a base station that performs at least some of the rendering of content for display and a device connected to the base station via a physical connection (i.e., a data communications cable), or stand-alone devices that perform rendering of content locally.
Stand-alone systems allow users freedom of movement; however, because of restraints including size, weight, batteries, and heat, stand-alone devices are generally limited in terms of computing power and thus limited in the quality of content that can be rendered. Apple says that “the base stations of tethered systems may provide more computing power and thus higher quality rendering than stand-alone devices; however, the physical cable tethers the device to the base station and thus constrains the movements of the user.” However, a wireless connection between Apple Glasses and a Mac base station could overcome such constraints.
Here’s Apple’s summary of the invention: “A mixed reality system that includes a device and a base station that communicate via a wireless connection The device may include sensors that collect information about the user’s environment and about the user. The information collected by the sensors may be transmitted to the base station via the wireless connection.
“The base station renders frames or slices based at least in part on the sensor information received from the device, encodes the frames or slices, and transmits the compressed frames or slices to the device for decoding and display. The base station may provide more computing power than conventional stand-alone systems, and the wireless connection does not tether the device to the base station as in conventional tethered systems. The system may implement methods and apparatus to maintain a target frame rate through the wireless link and to minimize latency in frame rendering, transmittal, and display.”
