360 degree multi-viewport system

Abstract

Some embodiments of a method may include: receiving metadata indicating which portion of an immersive video corresponds to a first viewport of the immersive video displayed on a first display; tracking an orientation of a second display relative to the first display; determining a second viewport of the immersive video synchronized in time and orientation relative to the immersive video displayed on the first display based on the metadata and the orientation of the second display relative to the first display; and displaying the second viewport on the second display.

1 . A method comprising:
 receiving metadata indicating which portion of a 360-degree video corresponds to a first viewport of the 360-degree video displayed on a fixed display;   tracking an orientation of a mobile device relative to the fixed display;   determining a second viewport of the 360-degree video synchronized in time and orientation relative to the 360-degree video displayed on the fixed display based on the metadata and the orientation of the mobile device relative to the fixed display; and   displaying the second viewport on the mobile device.   
     
     
         2 . The method of  claim 1 , wherein determining the second viewport comprises:
 determining a first angle from the orientation of the mobile device relative to the fixed display;   determining a second angle from a center of the first viewport of the 360-degree video displayed on the fixed display; and   orienting, for the second angle, the 360-degree video synchronized in time and space to generate the second viewport.   
     
     
         3 . The method of  claim 1 , wherein determining the second viewport of the 360-degree video comprises determining a virtual mirror viewport of the 360-degree video. 
     
     
         4 . The method of  claim 1 , wherein determining the second viewport of the 360-degree video comprises selecting a location of a virtual mirror corresponding to a portion of the 360-degree video displayed on the fixed display. 
     
     
         5 . The method of  claim 4 , wherein displaying the second viewport of the 360-degree video comprises rendering a portion of the 360-degree video corresponding to the location of the virtual mirror. 
     
     
         6 . The method of  claim 1 , further comprising:
 recording an image from a front-facing camera of a mobile device to generate a recorded image; and   overlaying the recorded image on the second viewport to generate an updated second viewport,   wherein displaying the second viewport displays the updated second viewport.   
     
     
         7 . The method of  claim 1 , further comprising:
 recording an image from a front-facing camera of a mobile device to generate a recorded image; and   displaying the recorded image as an overlay over the second viewport on the mobile device.   
     
     
         8 . The method of  claim 7 , wherein the second viewport is a virtual mirror of the first viewport. 
     
     
         9 . The method of  claim 1 , further comprising:
 selecting an object within the second viewport of the 360-degree video as a virtual mirror object,   wherein displaying the second viewport comprises replacing the virtual mirror object with a portion of the first viewport of the 360-degree video.   
     
     
         10 . The method of  claim 1 , further comprising:
 selecting an object within the 360-degree video as a virtual mirror object; and   replacing the virtual mirror object with a portion of the first viewport of the 360-degree video,   wherein displaying the second viewport comprises displaying the virtual mirror object replaced with the portion of the first viewport of the 360-degree video.   
     
     
         11 . The method of  claim 1 , wherein the second viewport is a virtual mirror of the first viewport. 
     
     
         12 . The method of  claim 1 , further comprising:
 recording an image from a front-facing camera of a mobile device to generate a recorded image;   determining a boundary of a user in the recorded image to generate a shape of the user;   cropping the recorded image by tracing the shape of the user to generate a cropped image; and   displaying the cropped image as an overlay over the second viewport on the mobile device.   
     
     
         13 . The method of  claim 1 , wherein determining the second viewport of the 360-degree video comprises:
 capturing an image of the 360-degree video displayed on the fixed display to generate a captured image;   determining a first time instant that the captured image is captured;   determining a second time instant that a first portion of the 360-degree video corresponding to the captured image is displayed;   determining a time delay as the difference between the first time instant and the second time instant;   time-adjusting the second viewport based on the time delay to generate a time-adjusted viewport; and   displaying the time-adjusted viewport on the mobile device.   
     
     
         14 . The method of  claim 1 , wherein tracking the orientation of the mobile device relative to the fixed display comprises:
 determining a first direction of the mobile device when the fixed display is parallel to the mobile device at a first time;   determining a second direction of the mobile device at a second time;   determining a heading difference between the first direction and the second direction; and   storing the heading difference as the orientation of the mobile device relative to the fixed display.   
     
     
         15 . The method of  claim 1 , wherein tracking the orientation of the mobile device relative to the fixed display comprises:
 determining a first direction of the mobile device relative to the fixed display at a first time;   determining a second direction of the mobile device relative to the fixed display at a second time;   determining a heading difference between the first direction and the second direction; and   storing the heading difference as the orientation of the mobile device relative to the fixed display.   
     
     
         16 . The method of  claim 1 , wherein the metadata comprises a media presentation description file. 
     
     
         17 . The method of  claim 1 , wherein the metadata indicates a change in spatial location of the first viewport of the 360-degree video. 
     
     
         18 . The method of  claim 1 , wherein receiving metadata indicating which portion of the 360-degree video corresponds to the first viewport of the 360-degree video displayed on the fixed display further comprises receiving metadata indicating the extent to which the portion of the 360-degree video corresponding to the first viewport of the 360-degree video is changing with time. 
     
     
         19 . An apparatus comprising:
 a processor; and   a non-transitory computer-readable medium storing instructions that are operative, when executed by the processor, cause the apparatus to:
 receive metadata indicating which portion of a 360-degree video corresponds to a first viewport of the 360-degree video displayed on a fixed display; 
 track an orientation of a mobile device relative to the fixed display; 
 determine a second viewport of the 360-degree video synchronized in time and orientation relative to the 360-degree video displayed on the fixed display based on the metadata and the orientation of the mobile device relative to the fixed display; and 
 display the second viewport on the mobile device. 
   
     
     
         20 . A method comprising:
 receiving an immersive video displayed with a first viewport on a first display;   receiving metadata indicating the first viewport of the immersive video;   tracking a relative orientation between the first display and a second display;   synchronizing an orientation of a second viewport of the immersive video relative to the first viewport to generate a synchronized immersive video for the second viewport;   determining a modified viewport of the synchronized immersive video based on the first viewport and the relative orientation; and   displaying the modified viewport on the second display.   
     
     
         21 - 26 . (canceled)