Apple patent involves ‘eye tracking’ features for ‘Apple Glasses’

Apple has been granted a patent (number 11,442,540) for “eye tracking using low resolution images.” It involves the rumored “Apple Glasses,” an augmented reality/virtual reality/mixed reality head-mounted display (HMD).

About the patent

Virtual reality (VR) allows users to experience and/or interact with an immersive artificial environment, such that the user feels as if they were physically in that environment. For example, virtual reality systems may display stereoscopic scenes to users in order to create an illusion of depth, and a computer may adjust the scene content in real-time to provide the illusion of the user moving within the scene. 

As Apple notes in the patent, when the user views images through a virtual reality system, the user may thus feel as if they are moving within the scenes from a first-person point of view. Similarly, mixed reality (MR) combines computer generated information (referred to as virtual content) with real world images or a real world view to augment, or add content to, a user’s view of the world. The simulated environments of VR and/or the mixed environments of MR may be utilized to provide an interactive user experience for multiple applications, such as applications that add virtual content to a real-time view of the viewer’s environment, interacting with virtual training environments, gaming, remotely controlling drones or other mechanical systems, viewing digital media content, interacting with the Internet, or the like. 

The eye tracker in Apple’s patent is a device for estimating eye positions and eye movement. Eye tracking systems have been used in research on the visual system, in psychology, psycholinguistics, marketing, and as input devices for human-computer interaction. In the latter application, typically the intersection of a person’s point of gaze with a desktop monitor is considered. 

Summary of the patent

Here’s Apple’s abstract of the patent: “Low-power eye tracking for detecting position and movements of a user’s eyes in a head-mounted device (HMD). An eye tracking system for an HMD may include eye tracking cameras. The eye tracking cameras may be used to capture low-resolution frames between capturing high-resolution frames, for example by binning pixels on the camera sensor or by capturing horizontal and vertical stripes or lines of pixels on the camera sensor rather than entire frames. This low-resolution information may be used to track relative movement of the user’s eyes with respect to the device in intervals between the processing of full, high-resolution frames captured by the eye tracking cameras.”

About Apple Glasses

When it comes to Apple Glasses, the rumors are abundant. Such a device will arrive in mid-to-late 2023. Or maybe 2024. It will be a head-mounted display. Or may have a design like “normal” glasses. Or it may be eventually be available in both. The Apple Glasses may or may not have to be tethered to an iPhone to work. Other rumors say that Apple Glasses could have a custom-build Apple chip and a dedicated operating system dubbed “rOS” for “reality operating system.”