An Apple Watch could one day be used to implement gesture controls on a Mac

Apple has filed for a patent (number 20210142214) for “machine-learning based gesture recognition using multiple sensors.” Among other things, it involves a gesture-based user interface for the Mac when a user is wearing an Apple Watch.

For example, a user may wish for a Mac to perform a particular action based on a gesture performed by the same hand wearing the smartwatch. In theory this would come in handy for gaming and virtually “swiping” through files in an app (such as pictures in Photos). Of course, the patent filing begs the question: would you need to have an Apple Watch on both wrists for total gesture recognition.

Here’s the summary of the patent filing; “A device implementing a system for machine-learning based gesture recognition includes at least one processor configured to, receive, from a first sensor of the device, first sensor output of a first type, and receive, from a second sensor of the device, second sensor output of a second type that differs from the first type.

“Then at least one processor is further configured to provide the first sensor output and the second sensor output as inputs to a machine learning model, the machine learning model having been trained to output a predicted gesture based on sensor output of the first type and sensor output of the second type. The at least one processor is further configured to determine the predicted gesture based on an output from the machine learning model, and to perform, in response to determining the predicted gesture, a predetermined action on the device.”