Apple patent involves using iPhones, iPads to film scenes that can generate 3D content

Apple has been granted a patent (number US 11089826 B2) for “Generating a Three-dimensional Model Using a Portable Electronic Device Recording.” 

About the patent

It involves the ability to turn recordings from devices such as iPhones and iPads into 3D models. The patent is directed to systems and methods for navigating a scene or around an object in three dimensions using deterministic movement of an electronic device. It also involves ystems and methods for generating a three-dimensional model of a scene or of an object for subsequent three-dimensional navigation by recording the scene using an electronic device.

In the patent Apple notes that users of electronic devices can view various information displayed by the devices. For example, a user can direct an electronic device to display a representation of a three-dimensional object (e.g., a photograph), or allow a user to navigate a representation of a location or of a virtual world (e.g., navigate through a series of images representing a city, such as a series of images in a street view of a mapping application). 

A user can direct the device to display such information using any suitable approach, including for example by selecting the information from a locally stored or remotely accessed source using an input mechanism. Once selected, the information can be displayed.

However, because a planar electronic device display can inherently be only a two-dimensional or planar display, the device can, at any given moment, provide only a partial representation of the displayed information. For example, when a three dimensional object is displayed, the electronic device can only display the object from a single perspective. 

To view other perspectives, the user may be required to select a different image representing the same object, or provide an input (e.g., selecting a button or dragging a mouse) to cause the object to rotate or spin, providing other perspectives for the object. Similarly, to navigate a representation of a three-dimensional environment, the electronic device can require the user to provide successive inputs using the input mechanism to move through the environment and change the displayed images to reflect the user’s movement through the environment.

In some cases, however, a user may not be able to provide an input to an input mechanism (e.g., the user’s hands are busy). Another mechanism may then be necessary to allow a user to view other portions of a displayed three-dimensional object or three-dimensional environment.

What’s more, Apple notes that users typically may not have an easy mechanism for generating three-dimensional representations of three-dimensional objects. Typically, such representations can require taking several images of an object from fixed and known positions and orientations relative the object, and subsequently processing the images to allow users to navigate between images to view all perspectives of the three-dimensional object or event. 

In particular, the user may be required to provide information related to the relative position and orientation of the lens for each image to allow for subsequent navigation of the images. This can be especially difficult without specialized equipment and prevent most users from generating three-dimensional models of objects or environments, and Apple wants to overcome such issues.

Summary of the patent

Here’s Apple’s abstract of the patent: “Systems and methods are provided for navigating a three-dimensional model using deterministic movement of an electronic device. An electronic device can load and provide an initial display of a three dimensional model (e.g., of an environment or of an object). As the user moves the electronic device, motion sensing components can detect the device movement and adjust the displayed portion of the three-dimensional model to reflect the movement of the device. 

“By walking with the device in the user’s real environment, a user can virtually navigate a representation of a three-dimensional environment. In some embodiments, a user can record an object or environment using an electronic device, and tag the recorded im and orientation of the programmable device relative to the three-dimensional object at the first time responsive to detection of movement ages or video with movement information describing the movement of the device during the recording. The recorded information can then be processed with the movement information to generate a three-dimensional model of the recorded environment or object.”

I hope that you’ll consider becoming a patron of Apple World Today. Prices range from $2 a month to $10 a month. You can sign up here. Thanks in advance for your support.