Generating three-dimensional video content from a set of images captured by a camera array

Abstract

An illustrative method includes at least one processor receiving video data that describes a set of images captured by a set of camera modules of a camera array, stitching the set of images together, based on relative positions of the camera modules, to generate three-dimensional (3D) video content, and correcting lens distortion to remove one or more lens distortion effects from the set of images or the 3D video content. Corresponding methods and systems are described.

What is claimed is: 
     
         1 . A system comprising:
 a non-transitory memory storing instructions; and   at least one processor communicatively coupled to the memory and configured to execute the instructions to:
 receive video data that describes a set of images captured by a set of camera modules of a camera array; 
 stitch the set of images together, based on relative positions of the camera modules, to generate three-dimensional (3D) video content; 
 correct lens distortion to remove one or more lens distortion effects from the set of images or the 3D video content. 
   
     
     
         2 . The system of  claim 1 , wherein:
 correcting lens distortion comprises identifying and removing the one or more lens distortion effects using a forward model to map a 3D real-world point onto an undistorted two-dimensional (2D) image.   
     
     
         3 . The system of  claim 1 , wherein:
 correcting lens distortion comprises identifying and removing the one or more lens distortion effects using an inverse model that is used to map a 3D real-world point from an undistorted two-dimensional (2D) image to a 3D image for a particular distance.   
     
     
         4 . The system of  claim 1 , wherein:
 correcting lens distortion is performed on the set of images prior to the set of images being stitched together.   
     
     
         5 . The system of  claim 4 , wherein:
 correcting lens distortion comprises identifying one or more of the images of the set of images as including the one or more lens distortion effects and removing the one or more lens distortion effects from the one or more images.   
     
     
         6 . The system of  claim 4 , wherein:
 correcting lens distortion is provided at one or more corners or one or more edges of one or more images of the set of images.   
     
     
         7 . The system of  claim 1 , wherein stitching the set of images together to generate the 3D video content is further based on a frame sync signal included in the video data that describes the set of images. 
     
     
         8 . The system of  claim 1 , wherein stitching the set of images together to generate the 3D video content is further based on a geometric calibration performed to determine the relative positions of the camera modules to one another. 
     
     
         9 . The system of  claim 8 , wherein the geometric calibration is performed after the video data is recorded by the camera modules and uses the video data to determine the relative positions of the camera modules to one another. 
     
     
         10 . A method comprising:
 receiving, by at least one processor, video data that describes a set of images captured by a set of camera modules of a camera array;   stitching, by the at least one processor, the set of images together, based on relative positions of the camera modules, to generate three-dimensional (3D) video content;   correcting, by the at least one processor, lens distortion to remove one or more lens distortion effects from the set of images or the 3D video content.   
     
     
         11 . The method of  claim 10 , wherein:
 correcting lens distortion comprises identifying and removing the one or more lens distortion effects using a forward model to map a 3D real-world point onto an undistorted two-dimensional (2D) image.   
     
     
         12 . The method of  claim 10 , wherein:
 correcting lens distortion comprises identifying and removing the one or more lens distortion effects using an inverse model that is used to map a 3D real-world point from an undistorted two-dimensional (2D) image to a 3D image for a particular distance.   
     
     
         13 . The method of  claim 10 , wherein:
 correcting lens distortion is performed on the set of images prior to the set of images being stitched together.   
     
     
         14 . The method of  claim 13 , wherein:
 correcting lens distortion comprises identifying one or more of the images of the set of images as including the one or more lens distortion effects and removing the one or more lens distortion effects from the one or more images.   
     
     
         15 . The method of  claim 13 , wherein:
 correcting lens distortion is provided at one or more corners or one or more edges of one or more images of the set of images.   
     
     
         16 . The method of  claim 10 , wherein stitching the set of images together to generate the 3D video content is further based on a frame sync signal included in the video data that describes the set of images. 
     
     
         17 . The method of  claim 10 , wherein stitching the set of images together to generate the 3D video content is further based on a geometric calibration performed to determine the relative positions of the camera modules to one another. 
     
     
         18 . The method of  claim 17 , wherein the geometric calibration is performed after the video data is recorded by the camera modules and uses the video data to determine the relative positions of the camera modules to one another. 
     
     
         19 . A non-transitory memory storing instructions executable by one or more processors to perform operations comprising:
 receiving video data that describes a set of images captured by a set of camera modules of a camera array;   stitching the set of images together, based on relative positions of the camera modules, to generate three-dimensional (3D) video content;   correcting lens distortion to remove one or more lens distortion effects from the set of images or the 3D video content.   
     
     
         20 . The non-transitory memory of  claim 19 , wherein:
 correcting lens distortion is performed on the set of images prior to the set of images being stitched together; and   correcting lens distortion comprises identifying one or more of the images of the set of images as including the one or more lens distortion effects and removing the one or more lens distortion effects from the one or more images.