The Apple Pencil could eventually work with the Vision Pro (via an iPad)

The Apple Pencil could eventually work with the Vision Pro, according to a newly granted patent (number US 11886650 B2) for a “Stylus-based Input System for a Head-mounted Device.” Input to the “spatial computer” would be accompanied by writing on an iPad.

About the patent

In the patent Apple notes that a head-mounted device (the Vision Pro, of course) can be worn by a user to display visual information within the field of view of the user. The head-mounted device can be used as a virtual reality (VR) system, an augmented reality (AR) system, and/or a mixed reality (MR) system. A user may observe outputs provided by the head-mounted device, such as visual information provided on a display. The display can optionally allow a user to observe an environment outside of the head-mounted device. Other outputs provided by the head-mounted device can include speaker output and/or haptic feedback. 

A user may further interact with the head-mounted device by providing inputs for processing by one or more components of the head-mounted device. For example, the user can provide tactile inputs, voice commands, and other inputs while the device is mounted to the user’s head. And Apple wants one of those tactile inputs to involve the Apple Pencil.

Summary of the patent

Here’s Apple’s abstract of the patent: “A tracking system includes a stylus comprising a light emitter, a head-mounted device comprising an optical sensor configured to detect the light emitter, and a processor communicatively coupled to the optical sensor and configured to determine a position of the stylus based on detection of the light emitter.”