You may one day be able to control Sir, Apple’s perssoal digital assistant, and your HomePod by looking at it. Apple has filed for a patent (number 20200103963) for “device control using gaze detection.”
In the patent filing, the tech giant notes that users provide inputs, such as key presses and voice inputs, to control electronic devices. For example, users activate a device’s button or speak a trigger phrase to start an application on the device. Such inputs frequently require the user to be within arm’s reach or within microphone range.
Intelligent automated assistants (or digital assistants) such as Siri can provide a beneficial interface between human users and electronic devices. Such assistants can allow users to interact with devices or systems using natural language in spoken and/or text forms. For example, a user can provide a speech input containing a user request to a digital assistant operating on an electronic device. Siri can interpret the user’s intent from the speech input, operationalize the user’s intent into a task, and perform the task. However, Apple says that sometimes performing tasks in this manner may be constrained in the manner by which a task is identified.
Apple’s solution? A HomePod (and, potentially, other devices such as Macs and iPads) with camera sensors that recognize when a user is eyeballing them and react accordingly. A camera may be used to periodically capture images of a user’s eyes. By processing the images to produce accurate orientation information reflecting the position of the user’s eyes relative to the device, the orientation of the device tracked by the motion sensor can be periodically updated.
Also, in some cases, however, a user may be limited to a particular set of commands such that the user cannot readily instruct a digital assistant to perform a task using natural-language speech inputs. What’s more, Apple says that digital assistants sometimes fail to adapt based on previous user behavior and in turn lack a desirable optimization of user experience.
Here’s the summary of the patent filing: “The present disclosure generally relates to controlling electronic devices. In some examples, the electronic device uses gaze information to activate a digital assistant. In some examples, the electronic device uses gaze information to identify an external device on which to act. In some examples, the electronic device provides an indication that distinguishes between different speakers.”
