Apple’s rumored “Apple Glasses” — an augmented reality (AR)/mixed reality (MR) head-mounted display — may be able to “render objects to match camera noise,” according to a new patent filing (number 202000273212).
Some augmented reality (AR) systems capture a video stream and combine images of the video stream with virtual content. Apple says the images of the video stream can be “very noisy,” especially in lower light condition in which certain ISO settings are used to boost image brightness. In contrast, since the virtual content renderer doesn’t suffer from the physical limitations of the image capture device, there’s little or no noise in the virtual content.
In the AR content that combines the images of real content with the virtual content, noise can be seen on the real objects but is generally missing from the virtual content. The lack of the appearance of noise, e.g., graininess, strength, its property variations between color channels, etc., on virtual content can result in the virtual content appearing to float, look detached, stand out, or otherwise fail to fit with the real content.
Apple says that existing systems and techniques don’t adequately account for image noise in presenting virtual content with image content in AR and other content that combines virtual content with real image content. The tech giant wants to change this with its Apple Glasses.
Here’s the summary of the patent filing: “Various implementations disclosed herein render virtual content with noise that is similar to or that otherwise better matches the noise found in the images with which the virtual content is combined. Some implementations involve identifying noise data for an image, creating a parameterized noise model based on the noise data, generating a noise pattern approximating noise of the image or another image using the parameterized noise model, and rendering content that includes the image and virtual content with noise added based on the noise pattern.”
When it comes to Apple Glasses, such a device will arrive next year or 2022, depending on which rumor you believe. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”
