Apple has been granted a patent (number 11,025,921) for “providing a virtual view by streaming serialized content.” The goal is to enhance a user’s viewing of live events — especially sports events — by easily offered alternate, even virtual views.
The patent would seem to involve Macs, iPhones, iPad, the Apple TV set-top box, and perhaps the rumored “Apple Glasses,” an augmented reality/virtual reality head-mounted display.
In the patent, Apple says that the ability view live events has improved significantly with the rise in technology. For example, in sporting events, venues are often outfitted with numerous cameras and sensors, which allows for improved user interaction with the game. TV cameras throughout the venue provide numerous views.
Spidercams allow cameras to pan vertically and horizontally over a playing field. These various cameras provide an enhanced user experience for people watching a game either at home, or in the venue itself. That is, the television feed of the game often incorporates replays from alternative camera views between plays or during timeouts. In addition, a user within the venue may often view an alternate view or replay on display screens within the venue. However, these types of video may require a large bandwidth to stream to a user.
Plus, Apple wants to take things to the next level. Part of the idea: a background and model of a moving object — such as a player or a ball — may be transmitted to a user device, along with artwork for the model, and other data. The user device may receive a serialized data stream indicating the movements of the model. Then, the user device may render the video stream on the user device using the background artwork model of the object, and movements of the model.
Apple says the result will be a smaller data stream because the background artwork isn’t being constantly transmitted. Only the changes to the scene are transmitted, and the scene is rendered at the user end. As an example, if a user is streaming a sports event, the field, or background, often doesn’t change.
All that changes is the movement of the players and the ball. This means that the background may be transmitted much less often than the players. What’s more, a central device may only transmit models of the players, and then transmit updates to the movement of the players as a serialized data stream. The user device may then render the display using the background art, the model of the moving object, and the received movements of the object.
But, wait, Apple’s not done. The company wants its devices to be able to give you extra info when you’re watching live events. For example, the previously mentioned serialized data stream may be used to transmit other data a user may request as ancillary information. If a user is streaming a football game, he/she may request statistics about the game. The serialized data stream may be modified to include the requested statistics, and the user device may modify the display by rendering the scene to include the statistics.
Here’s the summary of the patent: “Streaming image data includes receiving a request for a video stream of a scene of a live event, generate a background portion of the scene, obtaining a model of an object in the scene, transmitting the background portion and the model of the object, generating a serialized data stream indicating a modification of the object from the model, and transmitting the serialized data stream for the object.”
