Apple has been granted a patent (number 11,385,781) for “multi-display alignment through observed interactions.” The goal is for Macs and iPads with one or more external monitors attached to “learn” how you use them and align automatically.
About the patent
Often users of computing devices will have multiple display devices connected to their computing devices. For example, a user who works on a mobile computer (e.g., laptop computer, tablet computer, etc.) having a small display will often connect to a larger external display, additionally it is increasingly common for a portable device such as a mobile tablet computer to be used as an additional display for another system (such as Apple’s Sidecar feature for iPads to be used with Mac desktop or Laptops).
A user who works from a desktop computer may connect the desktop computer to multiple external displays. These additional displays may be connected via a cable or wirelessly, and, being hand held, their physical position, and thus expected logical arrangement with the other displays might change dynamically. This additional display area makes using the computing device much easier and more efficient when simultaneously working with multiple documents and/or multiple software applications on the computing device.
Apple says that when connecting a second or subsequent display device to a computing device that is already connected to a first display device, the user will often be required to explicitly configure the physical arrangement of the multiple display devices relative to each other so that when the user moves a cursor, window, or other graphical object, from one display device to another, or leaves an object spanning across multiple displays, the object moves smoothly and continuously from display to display as expected.
The user can then provide input to arrange the graphical representations of the display devices such that the representations mimic the physical alignment and/or relative positions of the actual, physical display devices. The computing device can then use the arrangement of the graphical representations of the display devices to determine how to align the display buffers for the display devices and/or how move a cursor from one display to another.
Apple wants a Mac or iPad to be able to use the arrangement of the graphical representations of the display devices to determine how to align the display buffers for the display devices and/or how move a cursor from one display to another.
Summary of the patent
Here’s Apple’s abstract of the patent: “In some implementations, a computing device can perform multi-display alignment through observed user interactions. The computing device can receive user input aligning a first alignment object on a first display device with a second alignment object on a second display device. The computing device can align the display buffers for each display device based on the positions of the alignment objects in each display buffer corresponding to each display device.
“The computing device can align display buffers based on observed movements of graphical objects between multiple display devices. When display buffers corresponding to the display devices are misaligned, the user may correct the path of a graphical object when moving the graphical object between display devices. The computing device can detect the correction and align the display buffers of the display devices so that graphical objects are presented at the appropriate locations when moved between the display devices.”
