Utilizing three overlapping images for exposure correction during panoramic image stitching

Abstract

A method for panorama exposure correction can begin with the receiving of a hemispherical-coverage panoramic digital image set by a panoramic image stitching system. The hemispherical-coverage panoramic digital image set can be comprised of a ceiling image and at least three side images that are all captured simultaneously. The images of the image set can be aligned to create a panoramic image. Based on the alignment, overlapping regions between two adjacent side images and the ceiling image can be identified, the placement of blending seams can be determined, and the region of each image affected by a exposure correction can be determined. The exposure correction for each identified overlapping region can be determined. The exposure of the affected region of each image can then be adjusted in accordance with the determined exposure correction. A gradient blending can be performed for a predefined area across the blending seams.

What is claimed is: 
     
       1. A method for panorama exposure correction comprising:
 receiving of a hemispherical-coverage panoramic digital image set by a panoramic image stitching system, wherein the hemispherical-coverage panoramic digital image set is captured by a hemispherical-coverage panoramic digital image capture device that utilizes a hemispherical panoramic camera module comprising at least three side digital cameras in fixed positions circumferentially-parallel with a base of the hemispherical panoramic camera module and one ceiling digital camera in a fixed upwards position perpendicular to the at least three circumferentially-parallel digital cameras, wherein a field of view of each side digital camera overlaps with the field of view of its neighboring side digital cameras and the field of view of the ceiling digital camera, wherein said hemispherical-coverage panoramic digital image set is then comprised of a ceiling image and at least three side images that are all captured simultaneously; 
 aligning images of the hemispherical-coverage panoramic digital image set to create a panoramic image; 
 based on said alignment, identifying overlapping regions between two adjacent side images and the ceiling image; 
 based on said alignment and identified overlapping regions, determining a placement of blending seams, wherein a blending seam is an artificial vertical line that acts as a boundary between exposure corrections performed for two overlapping regions, wherein the blending seam is placed at a halfway point between the two overlapping regions in a non-terminal side image, wherein the blending seams are extended onto the ceiling image; 
 based on said alignment and identified overlapping regions, determining a region of each image to be affected by a exposure correction of the overlapping region, wherein an affected region of a terminal side image is an entirety of the terminal side image, wherein the affected region of the non-terminal side image is horizontally bounded by an edge contained in the overlapping region and the blending seam of the non-terminal side image, wherein the affected region of the ceiling image is horizontally bounded by one of a terminating edge and the blending seam and two blending seams; 
 determining the exposure correction for each identified overlapping region, wherein the exposure correction normalizes exposure between the two adjacent side images and ceiling image; 
 adjusting a exposure of the affected region of each image contributing to the overlapping region in accordance with the determined exposure correction; and 
 performing a gradient blending for a predefined area across the blending seams. 
 
     
     
       2. The method of  claim 1 , wherein aligning of the hemispherical-coverage panoramic digital image set further comprises:
 identifying a starting image from the at least three side images, wherein the starting image represents a beginning of the final panorama; and 
 transforming the ceiling image from an original perspective into a panorama perspective, wherein sections of the ceiling image positioned along edges in the original perspective are all repositioned to a bottom edge in the panorama perspective, wherein features in the beginning of the ceiling image in the panorama perspective correspond to features in the starting image and features in the bottom edge in the panorama perspective correspond to features at a top edge of the at least three side images. 
 
     
     
       3. The method of  claim 2 , wherein transforming the digital image further comprises:
 applying at least one image processing function to the ceiling image to adjust for at least one of distortion, parallax, resolution differences, and lens differences. 
 
     
     
       4. The method of  claim 1 , wherein determining the exposure correction for each image further comprises:
 calculating a luminance matrix Z for pixels within the identified overlapping region that is a numerical average of luminance value matrices A, B, and C, wherein matrix A is a subset of a matrix I A  that expresses luminance values for pixels of a first side image within the identified overlapping region, matrix B is a subset of a matrix I B  that expresses luminance values for pixels of a second side image within the identified overlapping region, and matrix C is a subset of a matrix I C  that expresses luminance values for pixels of the ceiling image within the identified overlapping region, wherein matrices I A , I B , and I C  represent the luminance values for the affected regions of the first, second, and ceiling images, respectively; and 
 executing a histogram equalization upon the luminance matrix Z using a histogram equalization formula, wherein a matrix Z′ is a result of said histogram equalization, wherein the histogram equalization function is the exposure correction. 
 
     
     
       5. The method of  claim 4 , wherein adjusting the exposure of the affected region of each image further comprises:
 replacing matrices A, B, and C with matrix Z′ within matrices I A , I B , and I C , respectively; and 
 independently performing the histogram equalization upon each matrix I A , I B , and I C . 
 
     
     
       6. The method of  claim 5 , wherein the histogram equalization performed upon matrices I A , I B , and I C  is weighted based on a distance of a pixel from the overlapping region. 
     
     
       7. The method of  claim 4 , wherein calculating the luminance matrix Z further comprises:
 for each luminance value in matrices A, B, and C at a position (m, n), computing luminance differences between each unique luminance value set, wherein m and n are integer values bound by M and N, respectively, that define a size of the matrices A, B, and C; 
 comparing each luminance difference to an outlier threshold that represents a maximum degree of variation between luminance values; and 
 when both luminance differences for a luminance value exceed the outlier threshold, discarding the luminance value from the calculation. 
 
     
     
       8. The method of  claim 1 , wherein the hemispherical-coverage panoramic digital image capture device captures digital video, wherein the hemispherical-coverage panoramic digital image set is a set of video frames for an identical time index. 
     
     
       9. The method of  claim 1 , wherein the panoramic image stitching system is a software application that operates on one of the hemispherical-coverage panoramic digital image capture device and at least one computing device remotely-located from the hemispherical-coverage panoramic digital image capture device. 
     
     
       10. The method of  claim 1 , wherein receiving the hemispherical-coverage panoramic digital image set further comprises:
 ascertaining a colorspace of the hemispherical-coverage panoramic digital image set; and 
 when the ascertained colorspace is not a hue, saturation, and value (HSV) colorspace, converting the colorspace of the hemispherical-coverage panoramic digital image set to HSV. 
 
     
     
       11. A computer program product comprising a non-transitory computer readable storage medium having computer usable program code embodied therewith, the computer usable program code comprising:
 computer usable program code configured to receive a hemispherical-coverage panoramic digital image set that was captured by a hemispherical-coverage panoramic digital image capture device that utilizes a hemispherical panoramic camera module comprising at least three side digital cameras in fixed positions circumferentially-parallel with a base of the hemispherical panoramic camera module and one ceiling digital camera in a fixed upwards position perpendicular to the at least three circumferentially-parallel digital cameras, wherein a field of view of each side digital camera overlaps with the field of view of its neighboring side digital cameras and the field of view of the ceiling digital camera, wherein said hemispherical-coverage panoramic digital image set is then comprised of a ceiling image and at least three side images that are all captured simultaneously; 
 computer usable program code configured to align images of the hemispherical-coverage panoramic digital image set to create a panoramic image; 
 computer usable program code configured to, based on said alignment, identify overlapping regions between two adjacent side images and the ceiling image; 
 computer usable program code configured to, based on said alignment and identified overlapping regions, determine a placement of blending seams, wherein a blending seam is an artificial vertical line that acts as a boundary between exposure corrections performed for two overlapping regions, wherein the blending seam is placed at a halfway point between the two overlapping regions in a non-terminal side image, wherein the blending seams are extended onto the ceiling image; 
 computer usable program code configured to, based on said alignment and identified overlapping regions, determine a region of each image to be affected by a exposure correction of the overlapping region, wherein an affected region of a terminal side image is an entirety of the terminal side image, wherein the affected region of the non-terminal side image is horizontally bounded by an edge contained in the overlapping region and the blending seam of the non-terminal side image, wherein the affected region of the ceiling image is horizontally bounded by one of a terminating edge and the blending seam and two blending seams; 
 computer usable program code configured to determine the exposure correction for each identified overlapping region, wherein the exposure correction normalizes exposure between the two adjacent side images and ceiling image; 
 computer usable program code configured to adjust a exposure of the affected region of each image contributing to the overlapping region in accordance with the determined exposure correction; and 
 computer usable program code configured to perform a gradient blending for a predefined area across the blending seams. 
 
     
     
       12. The computer program product of  claim 11 , wherein the computer usable program code for aligning of the hemispherical-coverage panoramic digital image set further comprises:
 computer usable program code configured to identify a starting image from the at least three side images, wherein the starting image represents a beginning of the final panorama; and 
 computer usable program code configured to transform the ceiling image from an original perspective into a panorama perspective, wherein sections of the ceiling image positioned along edges in the original perspective are all repositioned to a bottom edge in the panorama perspective, wherein features in the beginning of the ceiling image in the panorama perspective correspond to features in the starting image and features in the bottom edge in the panorama perspective correspond to features at a top edge of the at least three side images. 
 
     
     
       13. The computer program product of  claim 11 , wherein the computer usable program code for determining the exposure correction for each image further comprises:
 computer usable program code configured to calculate a luminance matrix Z for pixels within the identified overlapping region that is a numerical average of luminance value matrices A, B, and C, wherein matrix A is a subset of a matrix I A  that expresses luminance values for pixels of a first side image within the identified overlapping region, matrix B is a subset of a matrix I B  that expresses luminance values for pixels of a second side image within the identified overlapping region, and matrix C is a subset of a matrix I C  that expresses luminance values for pixels of the ceiling image within the identified overlapping region, wherein matrices I A , I B , and I C  represent the luminance values for the affected regions of the first, second, and ceiling images, respectively; and 
 computer usable program code configured to execute a histogram equalization upon the luminance matrix Z using a histogram equalization formula, wherein a matrix Z′ is a result of said histogram equalization, wherein the histogram equalization function is the exposure correction. 
 
     
     
       14. The computer program product of  claim 13 , wherein the computer usable program code for adjusting the exposure of the affected region of each image further comprises:
 computer usable program code configured to replace matrices A, B, and C with matrix Z′ within matrices I A , I B , and I C , respectively; and 
 computer usable program code configured to independently perform the histogram equalization upon each matrix I A , I B , and I C . 
 
     
     
       15. The computer program product of  claim 14 , wherein the histogram equalization performed upon matrices I A , I B , and I C  is weighted based on a distance of a pixel from the overlapping region. 
     
     
       16. The computer program product of  claim 13 , wherein the computer usable program code for calculating the luminance matrix Z further comprises:
 computer usable program code configured to, for each luminance value in matrices A, B, and C at a position (m, n), compute luminance differences between each unique luminance value set, wherein m and n are integer values bound by M and N, respectively, that define a size of the matrices A, B, and C; 
 computer usable program code configured to compare each luminance difference to an outlier threshold that represents a maximum degree of variation between luminance values; and 
 computer usable program code configured to, when both luminance differences for a luminance value exceed the outlier threshold, discarding the luminance value from the calculation. 
 
     
     
       17. The computer program product of  claim 11 , wherein the computer usable program code for receiving the hemispherical-coverage panoramic digital image set further comprises:
 computer usable program code configured to ascertain a colorspace of the hemispherical-coverage panoramic digital image set; and 
 computer usable program code configured to, when the ascertained colorspace is not a hue, saturation, and value (HSV) colorspace, convert the colorspace of the hemispherical-coverage panoramic digital image set to HSV. 
 
     
     
       18. A panoramic image stitching system comprising:
 a hemispherical-coverage panoramic digital image set captured by a hemispherical-coverage panoramic digital image capture device that utilizes a hemispherical panoramic camera module comprising at least three side digital cameras in fixed positions circumferentially-parallel with a base of the hemispherical panoramic camera module and one ceiling digital camera in a fixed upwards position perpendicular to the at least three circumferentially-parallel digital cameras, wherein a field of view of each side digital camera overlaps with the field of view of its neighboring side digital cameras and the field of view of the ceiling digital camera, wherein said hemispherical-coverage panoramic digital image set is then comprised of a ceiling image and at least three side images that are all captured simultaneously; 
 a panoramic image stitching software application configured to generate a panoramic image from the hemispherical-coverage panoramic digital image set, said panoramic image stitching software application further comprises:
 a ceiling transformation module configured to transform the ceiling image from an original perspective to a panorama perspective; 
 a registration component configured to align adjacent side images with each other and the ceiling image, wherein the ceiling image is in the panorama perspective; and 
 a merging component configured to combine the hemispherical-coverage panoramic digital image set as aligned by the registration component, said merging component further comprises:
 an improved exposure correction handler configured to adjust a exposure of images in the hemispherical-coverage panoramic digital image set based upon luminance values in a region where two adjacent side images overlap with each other and the ceiling image. 
 
 
 
     
     
       19. The system of  claim 18 , wherein the hemispherical-coverage panoramic digital image capture device captures digital video, wherein the hemispherical-coverage panoramic digital image set is a set of video frames for an identical time index. 
     
     
       20. The system of  claim 18 , wherein the improved exposure correction handler further comprises:
 a colorspace conversion module to convert a colorspace of the hemispherical-coverage panoramic digital image set to a hue, saturation, and value (HSV) colorspace, when necessary; and 
 a histogram equalization module configured to perform a histogram equalization upon a set of input luminance values.