Apple is granted a patent for a ‘motion tracking user interface’

Apple has been granted a patent (number 9,715,273) for a “motion tracking user interface,” which doubtless involves its efforts in augmented and virtual reality. The company is already building augmented reality tools for developers into the upcoming iOS 11, thought this invention could apply to Macs as well as iPhones and iPads.

In the patent filing, Apple notes that input devices for computer systems commonly include a mouse, a keyboard, or other input devices (e.g., a pen-based device, a track ball, etc). Each of these input devices requires a user’s spare hand to operate. In some cases, a user may need both hands free to complete a task (e.g., typing on a keyboard) or the user may suffer from a disability that precludes the use of one hand. 

When the user desires to interact with the computer system, the user must interrupt the task to manipulate an input device. This may be difficult or impossible for folks with disabilities. Apple wants to remedy this by implementing motion tracking in its user interfaces.

Here’s Apple’s summary of the patent: “A method to transition focus of a display corresponding to an object’s motion tracked by a video camera or like device is disclosed. In one implementation, the display shows one or more windows or user interfaces on the display. The object’s motion can be used to select one of the windows or user interfaces on the display and manipulate content presented in the window or user interface. 

“In another implementations, the object’s motion can manipulate a three-dimensional graphical icon in a three-dimensional display environment, for example, by rotating it. In another implementation, the method further tracks motion of a second object and shifts focus of the display corresponding to the motion of the second object. In another implementation, a second display may be added to mirror the focus transition corresponding to the object’s motion.”

Earlier this year, Apple acquired SenSoMotoric Instruments (SMI), a Germany-based company that makes eye tracking hardware and software. SMI provides eye tracking systems for scientific research, professional solutions and original equipment manufacturer applications. 

The eye trackers can be combined with motion tracking systems, EEG, and other biometric data. They can be integrated into virtual reality environments, head-mounted displays (such as the Oculus Rift), simulators, cars, or computers as a measurement or interaction modality.

Of course, Apple files for — and is granted — lots of patents by the U.S. Patent & Trademark Office. Many are for inventions that never see the light of day. However, you never can tell which ones will materialize in a real product.