Predictive content delivery for video streaming services

Abstract

Methods, systems, and apparatuses for predicting a user field-of-view change and determining transmission regions of immersive video (IV) content for an IV device are presented. A field-of-view change of a user device may be predicted. A transmission region in one or more video frames of the IV content may be determined based on the predicted change.

The invention claimed is: 
     
       1. A method comprising:
 determining, by a computing device, a display area of a user device displaying video content comprising a plurality of video frames; 
 determining, based on one or more video frames of the plurality of video frames, a predicted change to the display area; 
 determining, based on the predicted change to the display area, a subarea of an image area of an upcoming video frame of the plurality of video frames; and 
 sending, to the user device, the subarea of the upcoming video frame. 
 
     
     
       2. The method of  claim 1 , wherein the subarea is greater than the display area and less than the image area of the upcoming video frame. 
     
     
       3. The method of  claim 1 , wherein determining the predicted change to the display area comprises determining a predicted direction of movement of the display area. 
     
     
       4. The method of  claim 1 , further comprising:
 determining, by the computing device, a visual object in the plurality of video frames, 
 wherein the visual object is located outside of the display area, and 
 wherein determining the subarea comprises extending an edge of the subarea toward the visual object. 
 
     
     
       5. The method of  claim 4 , wherein:
 the visual object comprises a probability value representing a probability that a field-of-view (FOV) will move toward the visual object during playback of the video content; and 
 wherein extending the edge of the subarea toward the visual object comprises increasing a size of the subarea based on the probability value. 
 
     
     
       6. The method of  claim 5 , further comprising:
 receiving, by the computing device, an indication of whether the FOV moved toward the visual object during playback of the video content; and 
 updating, by the computing device and based on the indication, the probability value. 
 
     
     
       7. The method of  claim 1 , further comprising:
 determining, by the computing device, an audio object associated with the plurality of video frames; and 
 wherein determining the predicted change to the display area comprises determining the predicted change to the display area based on the audio object. 
 
     
     
       8. The method of  claim 1 , further comprising:
 determining, by the computing device, a delay associated with the predicted change; and 
 wherein determining the subarea comprises determining a change of the subarea based on the delay. 
 
     
     
       9. An apparatus comprising:
 one or more processors; 
 memory storing instructions that, when executed by the one or more processors, cause the apparatus to:
 determine a display area of a user device displaying video content comprising a plurality of video frames; 
 determine, based on one or more video frames of the plurality of video frames, a predicted change to the display area; 
 determine, based on the predicted change to the display area, a subarea of an image area of an upcoming video frame of the plurality of video frames; and 
 send, to the user device, the subarea of the upcoming video frame. 
 
 
     
     
       10. The apparatus of  claim 9 , wherein the subarea is greater than the display area and less than the image area of the upcoming video frame. 
     
     
       11. The apparatus of  claim 9 , wherein the instructions, when executed by the one or more processors, cause the apparatus to determine the predicted change to the display area by determining a predicted direction of movement of the display area. 
     
     
       12. The apparatus of  claim 9 , wherein the instructions, when executed by the one or more processors, further cause the apparatus to:
 determine a visual object in the plurality of video frames, 
 wherein the visual object is located outside of the display area, and 
 determine the subarea of the image area of the upcoming video frame of the plurality of video frames by extending an edge of the subarea toward the visual object. 
 
     
     
       13. The apparatus of  claim 12 , wherein:
 the visual object comprises a probability value representing a probability that a field-of-view (FOV) will move toward the visual object during playback of the video content; and 
 wherein the instructions, when executed by the one or more processors, cause the apparatus to determine the subarea by extending the edge of the subarea toward the visual object by increasing a size of the subarea based on the probability value. 
 
     
     
       14. The apparatus of  claim 13 , wherein the instructions, when executed by the one or more processors, further cause the apparatus to:
 receive an indication of whether the FOV has moved toward the visual object during playback of the video content; and 
 update, based on the indication, the probability value. 
 
     
     
       15. The apparatus of  claim 9 , wherein the instructions, when executed by the one or more processors, further cause the apparatus to:
 determine an audio object associated with the plurality of video frames; and 
 determine the predicted change to the display area by determining the predicted change to the display area based on the audio object. 
 
     
     
       16. The apparatus of  claim 9 , wherein the instructions, when executed by the one or more processors, further cause the apparatus to:
 determine a delay associated with the predicted change; and 
 determine the subarea by determining a change of the subarea based on the delay. 
 
     
     
       17. A non-transitory computer-readable medium storing instructions that, when executed, cause:
 determining a display area of a user device displaying video content comprising a plurality of video frames; 
 determining, based on one or more video frames of the plurality of video frames, a predicted change to the display area; 
 determining, based on the predicted change to the display area, a subarea of an image area of an upcoming video frame of the plurality of video frames; and 
 sending, to the user device, the subarea of the upcoming video frame. 
 
     
     
       18. The non-transitory computer-readable medium of  claim 17 , wherein the subarea is greater than the display area and less than the image area of the upcoming video frame. 
     
     
       19. The non-transitory computer-readable medium of  claim 17 , wherein determining the predicted change to the display area comprises determining a predicted direction of movement of the display area. 
     
     
       20. The non-transitory computer-readable medium of  claim 17 , wherein the instructions, when executed, further cause:
 determining a visual object in the plurality of video frames, 
 wherein the visual object is located outside of the display area, and 
 wherein determining the predicted change comprises extending an edge of the subarea toward the visual object. 
 
     
     
       21. The non-transitory computer-readable medium of  claim 20 , wherein:
 the visual object comprises a probability value representing a probability that a field-of-view (FOV) will move toward the visual object during playback of the video content; and 
 wherein extending the edge of the subarea toward the visual object comprises increasing a size of the subarea based on the probability value. 
 
     
     
       22. The non-transitory computer-readable medium of  claim 21 , wherein the instructions, when executed, further cause:
 receiving an indication of whether the FOV moved toward the visual object during playback of the video content; and 
 updating, based on the indication, the probability value. 
 
     
     
       23. The non-transitory computer-readable medium of  claim 17 , wherein the instructions, when executed, further cause:
 determining an audio object associated with the plurality of video frames; and 
 wherein determining the predicted change to the display area comprises determining the predicted change to the display area based on the audio object. 
 
     
     
       24. The non-transitory computer-readable medium of  claim 17 , wherein the instructions, when executed, further cause:
 determining a delay associated with the predicted change, 
 wherein determining the subarea comprises determining a change of the subarea based on the delay.