Apple has filed for a patent (number US 20230076716 A1) for “multi device gesture control.” The goal is for you to be able to control your Macs, iPhones, iPads, HomePods, and more with just a wave.
About the patent filing
Electronic devices such as wearable electronic devices are often provided with input components such as keyboards, touchpads, touchscreens, or buttons that enable a user to interact with the electronic device. In some cases, an electronic device can be configured to accept a gesture input from a user for controlling the electronic device.
In the patent filing, Apple says that sensors for detecting gestures may be any sensors that generate input signals responsive to physical movements and/or positioning of a user’s hand, wrist, arm, and/or any other suitable portion of a user’s body. For example, to generate the input signals, the sensors may detect movement and/or positioning of external and/or internal structures of the user’s hand, wrist, and/or arm during the physical movements of the user’s hand, wrist, and/or arm.
For example, light reflected from or generated by the skin of the user can be detected by one or more cameras or other optical or infrared sensors. As another example, electrical signals generated by the muscles, tendons or bones of the wearer can be detected.
As still another example, ultrasonic signals generated by an electronic device and reflected from the muscles, tendons or bones of the user can be detected by an ultrasonic sensor. In general, EMG sensors, ultrasonic sensors, cameras, IMU sensors (e.g., an accelerometer, a gyroscope and/or a magnetometer), and/or other sensors may generate signals that can be provided to machine-learning models to identify a position or a motion of the wearer’s hand, wrist, arm, and/or other portion of the user’s body, and thereby detect user gestures.
Summary of the patent filing
Here’s Apple’s abstract of the patent filing: “Aspects of the subject technology relate to gesture-control inputs to an electronic device for controlling one or more other devices. The electronic device can efficiently provide gesture control for multiple other devices by mapping a finite set of user gestures to a specific set of gesture-control elements for each of the multiple other devices. In this way a single gesture can be detected for potentially controlling various different functions of various different devices. Prior to gesture control, the electronic device may receive a selection of a particular one of the multiple other devices for control, and obtain the specific set of gesture-control elements for gesture control of that selected device.”
