Apple granted patent for ‘On the Fly Enrollment for Facial Recognition’

Apple has been granted a patent (number US 11935327 B1) for “On the Fly Enrollment for Facial Recognition.” It involves its Face ID technology,

About the patent

The patent relates to methods and systems for face detection and recognition in images captured by a camera on a device. More particularly, it involves the implementation of new neural networks used in face detection and recognition processes during use of the device.

As Apple notes in the patent, biometric authentication processes are being used more frequently to allow users to more readily access their devices without the need for passcode or password authentication. One example of a biometric authentication process is fingerprint authentication using a fingerprint sensor. 

Facial recognition is another biometric process that may be used for authentication of an authorized user of a device. Facial recognition processes are generally used to identify individuals in an image and/or compare individuals in images to a database of individuals to match the faces of individuals.

For authentication using facial recognition, an authorized user typically follows an enrollment protocol to register the user’s face on the device for future unlocking of the device using facial recognition authentication. The enrollment protocol typically has the user follow a controlled regiment to capture the user’s face in different poses and/or positions to generate enrollment data in order to provide the best possible experience for the user in unlocking the device. Some systems and/or devices may store the enrollment data as raw user data (e.g., raw data for images of the user is stored on the device). 

Apple points out that storing raw user data for long term use as enrollment data may, however, be unsafe if information from the system or device is illegally accessed (e.g., stolen or hacked). To prevent these situations, certain systems and/or devices may store processed data as the enrollment data (e.g., data generated by processing images of the user as described herein). Storing processed data for enrollment data instead of raw user data may provide enhanced security for the user by eliminating the long term storage of raw user data on the device.

Apple says that software operating the facial recognition authentication on the device may often be updated as improvements in the models (e.g., neural networks) involved in the process are made. The tech giant wants its devices to provide an even better user experience by providing methods for seamlessly transitioning template information from the currently operation version of a device to the updated (new) model.

Summary of the patent

Here’s Apple’s abstract of the patent: “When a software update is provided to a device that implements a facial recognition authentication process, a new authentication algorithm to operate the facial recognition authentication process may be included as part of software update. For a period of time, the new authentication algorithm may operate a “virtual” facial recognition authentication process alongside operation of the existing facial recognition authentication process using the existing (e.g., earlier version) authentication algorithm. 

“The performance of the new authentication algorithm in providing facial recognition authentication (as assessed by the “virtual” process) may be compared to the performance of the existing authentication algorithm in providing facial recognition authentication during the period of time. When the performance of the new authentication algorithm is determined to have a satisfactory performance, operation of the actual facial recognition authentication process on the device may be switched to the new authentication algorithm.”