Imaging device and method for generating an undistorted wide view image

Abstract

Certain aspects of the technology disclosed herein involve combining images to generate a wide view image of a surrounding environment. Images can be recorded using an stand-alone imaging device having wide angle lenses and/or normal lenses. Images from the imaging device can be combined using methods described herein. In an embodiment, a pixel correspondence between a first image and a second image can be determined, based on a corresponding overlap area associated with the first image and the second image. Corresponding pixels in the corresponding overlap area associated with the first image and the second image can be merged based on a weight assigned to each of the corresponding pixels.

What is claimed is: 
     
         1 . An imaging device for generating an undistorted image of a surrounding environment, the imaging device comprising:
 a first plurality of cameras comprising a plurality of fisheye lenses distributed around an outer surface of the imaging device,   wherein the first plurality of cameras is configured to record a first plurality of images associated with the surrounding environment, wherein each image in the first plurality of images comprises a first image periphery distorted by a fisheye lens in the plurality of fisheye lenses, and a first image center undistorted by the plurality of fisheye lenses;   a second plurality of cameras comprising a plurality of lenses distributed around the outer surface of the imaging device,   wherein the second plurality of cameras is configured to record a second plurality of images associated with the surrounding environment, wherein each image in the second plurality of images comprises a second image center undistorted by a camera in the second plurality of cameras, wherein the second plurality of image centers associated with the second plurality of images overlaps the first plurality of image peripheries associated with the first plurality of images; and   a processor disposed within the imaging device configured to perform a method comprising:
 receive the first plurality of images and the second plurality of images; 
 dewarp the first plurality of images; 
 determine a pixel correspondence between a first image in the first plurality of images and a second image in the second plurality of images, based on a corresponding overlap area associated with the first image and the second image; and 
 combine the first plurality of images and the second plurality of images to obtain an undistorted image associated with the surrounding environment. 
   
     
     
         2 . The method of  claim 1 , further comprising:
 obtaining a plurality of mapping functions corresponding to the at least one of the plurality of fisheye lenses, wherein a mapping function in the plurality of mapping functions comprises a transformation, wherein when the transformation is applied to the distorted image associated with the at least one of the plurality of fisheye lenses, the transformation dewarping the distorted image and producing an undistorted image.   
     
     
         3 . The method of  claim 2 , wherein said obtaining the plurality of mapping functions comprises:
 recording a first image using the at least one of the plurality of fisheye lenses;   recording a second image using at least one of the plurality of lenses, wherein the at least one of the plurality of lenses is disposed a corresponding orientation as the at least one of the plurality of fisheye lenses;   calculating the mapping function; and   storing the mapping function in a memory device.   
     
     
         4 . The method of  claim 2 , wherein said obtaining the plurality of mapping functions comprises retrieving the plurality of mapping functions from a memory device. 
     
     
         5 . The method of  claim 1 , wherein the said determining the pixel correspondence comprises:
 identifying a plurality of overlap areas associated with the first image in the first plurality of images and the second image in the second plurality of images;   calculating a plurality of first match scores corresponding to the plurality of overlap areas;   identifying an overlap area of the plurality of overlap areas having a highest first match score of the plurality of first match scores as the corresponding overlap area;   identifying one or more first set of pixels in the corresponding overlap area of the first image and one or more second set of pixels in the corresponding overlap area of the second image;   calculating a plurality of second match scores corresponding to the one or more first set of pixels and the one or more second set of pixels; and   identifying the one or more first sets of pixels and the one or more second sets of pixels having a highest second match score of the plurality of second match scores as corresponding pixels.   
     
     
         6 . The method of  claim 1 , further comprising:
 compressing the undistorted image by storing image data of an identified object in a database and inserting a reference to the stored image data of the identified object in the database; and   encrypting the inserted reference to the stored image data of the identified object.   
     
     
         7 . The method of  claim 1 , further comprising:
 identifying an object of a class in an image of the first plurality of images or the second plurality of images.   
     
     
         8 . The method of  claim 7 , further comprising:
 removing the identified object from the image by merging pixels of the identified object with corresponding pixels of a corresponding overlap area, wherein the pixels of the identified object are assigned a weight of 0.   
     
     
         9 . The method of  claim 7 , further comprising:
 locally dewarping a region surround the identified object by mapping corresponding pixels of the identified object in the first image and the second image to determine a distortion function and utilizing the distortion function to normalize the region surrounding the identified object.   
     
     
         10 . The method of  claim 1 , further comprising:
 identifying an object of a class in an image of the first plurality of images or the second plurality of images recorded by a camera of the first plurality of cameras or the second plurality of cameras.   
     
     
         11 . The method of  claim 10 , further comprising:
 determining a location of the identified object in another image recorded by another camera; and   focusing on the determined location of the identified object in the another image recorded by the another camera.   
     
     
         12 . An imaging device capable of recording an undistorted full view of a surrounding environment, the imaging device comprising:
 a first plurality of cameras comprising a plurality of fisheye lenses corresponding to the first plurality of cameras, wherein the first plurality of cameras is configured to record a first plurality of images associated with the surrounding environment, wherein each image in the first plurality of images comprises a first image periphery distorted by a fisheye lens in the plurality of fisheye lenses, and a first image center undistorted by the fisheye lens;   a second plurality of cameras comprising a plurality of lenses corresponding to the second plurality of cameras, wherein the second plurality of cameras is configured to record a second plurality of images associated with the surrounding environment, wherein any of the second plurality of images is undistorted by the plurality of lenses,   wherein a first area of any the first plurality of images overlaps a second area of any of the second plurality of images; and   a processor configured to combine the first plurality of images and the second plurality of images into an undistorted wide field of view image based on a corresponding overlap of the first and second areas.   
     
     
         13 . The imaging device of  claim 12 , wherein the first plurality of cameras and the second plurality of cameras are configured to record the first plurality of images and the second plurality of images synchronously. 
     
     
         14 . The imaging device of  claim 12 , wherein the processor is further configured to:
 receive the first plurality of images and the second plurality of images;   dewarp the first plurality of images; and
 determine a pixel correspondence between a first image in the first plurality of images and a second image in the second plurality of images to identify the corresponding overlap of the first and second areas. 
   
     
     
         15 . An imaging device capable of recording an undistorted full view of a surrounding environment, the imaging device comprising:
 a first plurality of cameras comprising a plurality of fisheye lenses distributed around the first plurality of cameras, wherein the first plurality of cameras is configured to record a first plurality of images associated with the surrounding environment, wherein each image in the first plurality of images comprises a first image periphery distorted by a fisheye lens in the plurality of fisheye lenses, and a first image center undistorted by the fisheye lens;   a second plurality of cameras comprising a plurality of lenses distributed around the second plurality of cameras, wherein the second plurality of cameras is configured to record a second plurality of images associated with the surrounding environment, wherein each image in the second plurality of images comprises a second image center undistorted by a camera in the second plurality of cameras, wherein the second plurality of image centers associated with the second plurality of images overlaps the first plurality of image peripheries associated with the first plurality of images; and   a processor configured to combine the first plurality of images and the second plurality of images into an undistorted wide field of view image.   
     
     
         16 . The imaging device of  claim 15 , wherein the first plurality of cameras and the second plurality of cameras are configured to record the first plurality of images and the second plurality of images synchronously. 
     
     
         17 . The imaging device of  claim 15 , wherein the processor is further configured to:
 receive the first plurality of images and the second plurality of images;   dewarp the first plurality of images; and
 determine a pixel correspondence between a first image in the first plurality of images and a second image in the second plurality of images to identify a corresponding overlap area. 
   
     
     
         18 . The imaging device of  claim 15 , wherein combining the first plurality of images and the second plurality of images comprises:
 merging corresponding pixels in a corresponding overlap area, said merging comprising:   for each pixel in the first image associated with the corresponding overlap area, assigning a weight ranging from 0 to 1 inclusive, and wherein the first weight is associated with a determined distortion of a pixel in the first image; and   for each corresponding pixel in the second images associated with the corresponding overlap area, weighing the pixel by a second weight, wherein the second weight increases as the first weight decreases;   wherein the weight dictates a proportional contribution of each pixel to values of a merged pixel; and   tiling the first portion, the second portion, and the overlap portion to obtain the image undistorted by the plurality of fisheye lenses.   
     
     
         19 . The imaging device of  claim 15 , comprising a swivel configured to dispose the first plurality of cameras to assume a first position, wherein the first position enables the first plurality of cameras to record a third plurality of images, and wherein the processor is further configured to combine the first plurality of images, the second plurality of images, and the third plurality of images into the undistorted wide field of view image. 
     
     
         20 . The imaging device of  claim 15 , wherein the imaging device comprises a mobile device accessory configured to attach to a mobile device, and wherein the processor is configured to communicate with the mobile device.