A new patent filing for “Handheld Input Devices” show Apple is investigating ways for an Apple Pencil to interact with a Vision Pro.
About the patent filing
This combo would allow the Apple Pencil to be used with drawing apps on the Vision Pro, such as Freeform and Pixelmator. One possibility is that users would be able to draw with the Apple Pencil on a desk or another flat surface, and the drawing would appear in visionOS.
This would essentially turn a person’s surroundings into a giant canvas, complete with pressure and tilt sensitivity. And the patent filing notes that the Apple Pencil could be attached to the Vision Pro via a lanyard.
This isn’t the first hint of an Apple Pencil/Vision Pro collaboration. In January Apple was granted a patent for a “Stylus-based Input System for a Head-mounted Device.” Input to the “spatial computer” would be accompanied by writing on an iPad.
In the patent Apple noted that a head-mounted device (the Vision Pro, of course) can be worn by a user to display visual information within the field of view of the user. The head-mounted device can be used as a virtual reality (VR) system, an augmented reality (AR) system, and/or a mixed reality (MR) system. A user may observe outputs provided by the head-mounted device, such as visual information provided on a display. The display can optionally allow a user to observe an environment outside of the head-mounted device. Other outputs provided by the head-mounted device can include speaker output and/or haptic feedback.
A user may further interact with the head-mounted device by providing inputs for processing by one or more components of the head-mounted device. For example, the user can provide tactile inputs, voice commands, and other inputs while the device is mounted to the user’s head. And Apple wants one of those tactile inputs to involve the Apple Pencil.
Summary of the patent filing
Here’s Apple’s abstract of the patent filing: “A system may include an electronic device such as a head-mounted device and a handheld input device for controlling the electronic device. A lanyard may be removably attached to the handheld input device or a non-electronic object. The lanyard may include visual markers, such as infrared light-emitting diodes and/or fiducials, that can be detected by an external camera and used to track a location of the lanyard.
“For example, the lanyard may be fabric, and the visual markers may be incorporated into the fabric or attached to the fabric. The lanyard may also include motion sensors, visual-inertial odometry cameras, or other sensors to determine the location of the lanyard. The lanyard may be electrically coupled to the handheld input device, such as to transfer power and/or data. Alternatively, the lanyard may be coupled to a non-electronic object and may include a battery and/or a haptic output component.”
I hope you’ll help support Apple World Today by becoming a patron. Patreon pricing ranges from $2 to $10 a month. Thanks in advance for your support.
