Image stitching in a multi-camera array

Abstract

Images captured by multi-camera arrays with overlap regions can be stitched together using image stitching operations. An image stitching operation can be selected for use in stitching images based on a number of factors. An image stitching operation can be selected based on a view window location of a user viewing the images to be stitched together. An image stitching operation can also be selected based on a type, priority, or depth of image features located within an overlap region. Finally, an image stitching operation can be selected based on a likelihood that a particular image stitching operation will produce visible artifacts. Once a stitching operation is selected, the images corresponding to the overlap region can be stitched using the stitching operation, and the stitched image can be stored for subsequent access.

What is claimed is: 
     
         1 . A method for stitching images, comprising:
 accessing a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;   determining a location of a view window within the accessed set of images, the view window comprising a portion of one or more of the set of images being displayed to the user;   determining a distance between the view window and the overlap region;   responsive to the determined distance exceeding a threshold distance, performing a first stitching operation on the set of images; and   responsive to the determined distance being less than the threshold distance, performing a second stitching operation on the set of images, the second stitching operation comprising a higher quality stitching operation than the first stitching operation.   
     
     
         2 . The method of  claim 1 , wherein the set of images comprises images captured by a multi-camera array, each image in the set of images captured by a different camera in the multi-camera array. 
     
     
         3 . The method of  claim 1 , wherein the set of images comprises preview images for cameras in a multi-camera array, each preview image comprising an image representative of a current view of a camera in the multi-camera array, the view window comprising an electronic view finder for the multi-camera array. 
     
     
         4 . The method of  claim 1 , wherein the first stitching operation comprises averaging image information for a portion of each image corresponding to the overlap region. 
     
     
         5 . The method of  claim 1 , wherein the second stitching operation comprises determining a depth for each of one or more image features in the overlap region and applying a warp to a portion of each image corresponding to the overlap region based on the determined depths. 
     
     
         6 . The method of  claim 1 , further comprising:
 responsive to the determined distance being less than the threshold distance and greater than a second threshold distance, performing one or more pre-processing operations on a portion of each image corresponding to the overlap region, the pre-processing operations corresponding to the second stitching operation.   
     
     
         7 . A system for stitching images, comprising:
 an input configured to access a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;   an interface configured to determine a location of a view window within the accessed set of images, the view window comprising a portion of one or more of the set of images being displayed to the user; and   an image processing system comprising at least one hardware processor, the image processing system configured to:
 determine a distance between the view window and the overlap region; 
 responsive to the determined distance exceeding a threshold distance, perform a first stitching operation on the set of images; and 
 responsive to the determined distance being less than the threshold distance, perform a second stitching operation on the set of images, the second stitching operation comprising a higher quality stitching operation than the first stitching operation. 
   
     
     
         8 . The system of  claim 7 , wherein the set of images comprises images captured by a multi-camera array, each image in the set of images captured by a different camera in the multi-camera array. 
     
     
         9 . The system of  claim 7 , wherein the set of images comprises preview images for cameras in a multi-camera array, each preview image comprising an image representative of a current view of a camera in the multi-camera array, the view window comprising an electronic view finder for the multi-camera array. 
     
     
         10 . The system of  claim 7 , wherein the first stitching operation comprises averaging image information for a portion of each image corresponding to the overlap region. 
     
     
         11 . The system of  claim 7 , wherein the second stitching operation comprises determining a depth for each of one or more image features in the overlap region and applying a warp to a portion of each image corresponding to the overlap region based on the determined depths. 
     
     
         12 . The system of  claim 7 , the image processing system further configured to:
 responsive to the determined distance being less than the threshold distance and greater than a second threshold distance, perform one or more pre-processing operations on a portion of each image corresponding to the overlap region, the pre-processing operations corresponding to the second stitching operation.   
     
     
         13 . A non-transitory computer-readable storage medium storing executable computer instructions for stitching images, the instructions configured to, when executed by a processor, perform steps comprising:
 accessing a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;   determining a location of a view window within the accessed set of images, the view window comprising a portion of one or more of the set of images being displayed to the user;   determining a distance between the view window and the overlap region;   responsive to the determined distance exceeding a threshold distance, performing a first stitching operation on the set of images; and   responsive to the determined distance being less than the threshold distance, performing a second stitching operation on the set of images, the second stitching operation comprising a higher quality stitching operation than the first stitching operation.   
     
     
         14 . The computer-readable storage medium of  claim 13 , wherein the set of images comprises images captured by a multi-camera array, each image in the set of images captured by a different camera in the multi-camera array. 
     
     
         15 . The computer-readable storage medium of  claim 13 , wherein the set of images comprises preview images for cameras in a multi-camera array, each preview image comprising an image representative of a current view of a camera in the multi-camera array, the view window comprising an electronic view finder for the multi-camera array. 
     
     
         16 . The computer-readable storage medium of  claim 13 , wherein the first stitching operation comprises averaging image information for a portion of each image corresponding to the overlap region. 
     
     
         17 . The computer-readable storage medium of  claim 13 , wherein the second stitching operation comprises determining a depth for each of one or more image features in the overlap region and applying a warp to a portion of each image corresponding to the overlap region based on the determined depths. 
     
     
         18 . The computer-readable storage medium of  claim 13 , the instructions configured to, when executed, perform further steps comprising:
 responsive to the determined distance being less than the threshold distance and greater than a second threshold distance, performing one or more pre-processing operations on a portion of each image corresponding to the overlap region, the pre-processing operations corresponding to the second stitching operation.   
     
     
         19 . A method for stitching images, comprising:
 accessing a set of images associated with an overlap region for display to a user, the overlap region comprising a corresponding portion of each of the set of images;   determining a location of a view window within the accessed set of images, the view window comprising a portion of one or more of the set of images being displayed to the user;   determining a distance between the view window and the overlap region; and   responsive to the determined distance being less than a threshold distance, performing a stitching operation on the set of images to produce a stitched image.   
     
     
         20 . The method of  claim 19 , further comprising:
 responsive to the determined distance exceeding the threshold distance, refraining from performing a stitching operation on the set of images.