Analysis and manipulation of images and video for generation of surround views

Abstract

Various embodiments of the present invention relate generally to systems and methods for analyzing and manipulating images and video. According to particular embodiments, the spatial relationship between multiple images and video is analyzed together with location information data, for purposes of creating a representation referred to herein as a surround view. In particular embodiments, the surround view reduces redundancy in the image and location data, and presents a user with an interactive and immersive viewing experience.

What is claimed is: 
     
         1 . A method comprising:
 obtaining a plurality of images having location information, wherein the plurality of images include at least a portion of overlapping subject matter;   fusing the plurality of images into content and context models, wherein the content model includes an object and the context model includes scenery surrounding the object;   applying an enhancement algorithm to the content and context models; and   generating a surround view from the content and context models, wherein the surround view is a multi-view interactive digital media representation.   
     
     
         2 . The method of  claim 1 , wherein the plurality of images is obtained from a plurality of users. 
     
     
         3 . The method of  claim 1 , wherein the enhancement algorithm includes one or more of automatic frame selection, stabilization, view interpolation, filters, and compression. 
     
     
         4 . The method of  claim 1 , wherein the enhancement algorithm can be applied while the plurality of images is obtained. 
     
     
         5 . The method of  claim 1 , wherein the plurality of images includes images with different temporal information. 
     
     
         6 . The method of  claim 1 , wherein the plurality of images represents moving objects. 
     
     
         7 . The method of  claim 1 , wherein the surround view is a locally concave surround view. 
     
     
         8 . The method of  claim 1 , wherein the surround view is a locally convex surround view. 
     
     
         9 . The method of  claim 1 , wherein the surround view is a locally flat surround view. 
     
     
         10 . A system comprising:
 an input interface configured to obtain a plurality of images having location information, wherein the plurality of images include at least a portion of overlapping subject matter;   a processor configured to fused the plurality of images into content and context models, wherein the content model includes an object and the context model includes scenery surrounding the object, wherein the processor is further configured to apply an enhancement algorithm to the content and context models; and   an output interface configured to provide a surround view generated from the content and context models, wherein the surround view is a multi-view interactive digital media representation.   
     
     
         11 . The system of  claim 10 , wherein the plurality of images is obtained from a plurality of users. 
     
     
         12 . The system of  claim 10 , wherein the enhancement algorithm includes one or more of automatic frame selection, stabilization, view interpolation, filters, and compression. 
     
     
         13 . The system of  claim 10 , wherein the enhancement algorithm can be applied while the plurality of images is obtained. 
     
     
         14 . The system of  claim 10 , wherein the plurality of images includes images with different temporal information. 
     
     
         15 . The system of  claim 10 , wherein the plurality of images represents moving objects. 
     
     
         16 . The system of  claim 10 , wherein the surround view is a locally concave surround view. 
     
     
         17 . The system of  claim 10 , wherein the surround view is a locally convex surround view. 
     
     
         18 . The system of  claim 10 , wherein the surround view is a locally flat surround view. 
     
     
         19 . A computer readable medium comprising:
 computer code for obtaining a plurality of images having location information, wherein the plurality of images include at least a portion of overlapping subject matter;   computer code for fusing the plurality of images into content and context models, wherein the content model includes an object and the context model includes scenery surrounding the object;   computer code for applying an enhancement algorithm to the content and context models; and   computer code for generating a surround view from the content and context models, wherein the surround view is a multi-view interactive digital media representation.   
     
     
         20 . The computer readable medium of  claim 19 , wherein the plurality of images is obtained from a plurality of users.