Apple has filed for yet another patent (number 20190272650) — one of a dozen or so — that hints at future iterations of macOS and iOS devices that would allow you to control some user interface elements with your eyes, perhaps in conjunction with the rumored “Apple Glasses,” a head-mounted, augmented reality/virtual reality headset.
For determining gaze direction eye trackers can be used. Eye trackers observe features of the eye like the pupil, the limbus, blood vessels on the sclera, the eyeball or reflections of light sources (corneal reflections) in order to calculate the direction of the gaze.
Apple says this gaze direction is then mapped to an image of the scene captured by a head-mounted scene camera or a scene camera at any fixed location. The head-mounted scene camera (the Apple Glasses) is fixed with respect to the head, and therefore such a mapping can be performed, once a corresponding calibration has been executed. For performing the calibration a user may have to gaze at several defined points in the scene image captured by the head-mounted camera.
By using the correspondingly detected gaze directions the calibration can be performed resulting in a transformation which maps a gaze direction to a corresponding point in the scene image. According to Apple, in this approach any kind of eye tracker can be used if it allows mapping the gaze direction into images of a head-mounted scene camera. The company adds that this approach enables the determination of a gaze point in the scene image as taken by the head-mounted scene camera.
Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.
