Apple patent filing involves enhancing macOS’s Sidecar feature for the vision impaired

Apple has filed for a patent (number 20200264764) for “systems and methods for enabling low-vision users to interact with a touch-sensitive secondary display.” The goal is, apparently, to beef up the features of the company’s Sidecar feature to allow Mac users with vision problems to more effectively use an iPad as a secondary display.

Sidecar, introduced with macOS Catalina, allows users to extend their Mac desktop by using their iPad as a second display or as a high-precision input device across creative Mac apps. 

In the patent filing, Apple notes that vision problems can prevent many users from appreciating and using features available through touch-sensitive input devices that are also used to display affordances. For example, users of touch-sensitive secondary displays that may be located above a physical keyboard may not be able to view certain affordances because their fingers are occluding or covering up the affordances while they are displayed at a secondary display. 

What’s more, text and images in such secondary displays are often small. These problems are particularly acute for low-vision users, who may have difficulties seeing certain affordances that are displayed at a secondary display, and these difficulties are worsened and amplified by the impaired vision.

Apple’s idea is for an iPad “Sidecar’d” to a Mac to seamlessly offer a zoomed-in display of the Mac’s screen in accessibility mode. The tablet’s screen would sport a user interface that includes: application-specific “affordances” (possible interactions with the iPad display) and a system-level affordance, where each application-specific affordance and the system-level affordance are displayed with a first display size. 

Apple says that displaying a zoomed-in representation of at least one affordance of the application-specific affordance improves operability of the computing system, because low-vision users are able to interact with controls available at the touch-sensitive secondary display that may be too small (or may be occluded from view because a user’s finger is covering up the displayed controls) for the low-vision users to view accurately. In this way, low-vision users are able to take advantage of an improved man-machine interface by, e.g., having sustained interactions with a touch-sensitive secondary display (instead of having to constantly correct erroneous inputs).