Apple has been granted a patent (number 11524401) for “learning skills from video demonstrations.” But not for humans only, but also robots.
About the patent
The patent relates to the field of machine learning. In the patent Apple notes that humans are capable of performing complex physical tasks, while robots must be trained to perform these types of tasks. Reinforcement techniques that simulate performance of tasks by robots have been used to train robots.
However, Apple says that mmultiple types of errors (e.g., modeling errors) cause the conditions in a simulation to differ from real-world conditions. This can cause models that are successful in performing tasks in a simulation to fail when they are used to control real-world robotic systems, in what is commonly referred to as the sim-to-real gap.
Summary of the patent
Here’s Apple’s abstract of the patent: “A method includes determining motion imitation information for causing a system to imitate a physical task using a first machine learning model that is trained using motion information that represents a performance of the physical task, determining a predicted correction based on the motion information and a current state from the system using a second machine learning model that is trained using the motion information, determining an action to be performed by the system based on the motion imitation information and the predicted correction; and controlling motion of the system in accordance with the action.”
