Image frame selection for multi-frame fusion

Abstract

Methods, systems, apparatuses, and computer-readable storage mediums described herein are configured to fuse image frames captured by a camera to generate an image frame having high dynamic range and reduced amount of noise. For instance, after a shutter button of the camera is activated, one or more history image frames captured during a preview mode of the camera (e.g., before the shutter button is activated) are compared to a highlight recovery frame captured after the shutter button is activated to determine a level of similarity therebetween. The history image frame(s) may be captured with a first exposure value, and the highlight recovery frame may be captured with a second exposure value that is different than the first exposure value. History image frame(s) that are determined to be relatively similar to the highlight recovery frame are combined with the highlight recovery frame to generate the image frame having high dynamic range.

What is claimed is: 
     
       1. A system, comprising:
 a memory; 
 an image sensor configured to capture one or more first image frames during a preview mode of a camera; 
 at least one processor circuit configured to:
 detect activation of a shutter button of the camera; and 
 responsive to detecting activation of the shutter button, capture a second image frame; and 
 
 an image fusion engine configured to:
 determine that a level of similarity between a first image frame of the one or more first image frames and the second image frame meets a threshold condition; 
 responsive to a determination that the level of similarity between the first image frame of the one or more first image frames and the second image frame meets the threshold condition, combine the first image frame with the second image frame to generate a final image frame; and 
 store the final image frame in the memory. 
 
 
     
     
       2. The system of  claim 1 , wherein the system is included in a mobile computing device. 
     
     
       3. The system of  claim 1 , wherein the system is included in a dedicated camera device. 
     
     
       4. The system of  claim 1 , wherein the one or more first image frames are stored in a ring buffer of the system. 
     
     
       5. The system of  claim 4 , wherein the image fusion engine is further configured to:
 during the preview mode of the camera:
 compare a third image frame of the one or more first image frames stored in the ring buffer to a fourth image frame of the one or more first image frames stored in the ring buffer, the fourth image frame being stored in the ring buffer at a time prior to when the third image frame is stored in the ring buffer; 
 determine that said comparing does not result in a level of similarity that meets the threshold condition; and 
 responsive to a determination that said comparing does not result in a level of similarity that meets the threshold condition, discard the fourth image frame. 
 
 
     
     
       6. The system of  claim 5 , wherein the third image frame is the first image frame. 
     
     
       7. The system of  claim 4 , wherein the image fusion engine is configured to determine that the level of similarity between a first image frame of the one or more first image frames and the second image frame meets the threshold condition by:
 for each first image frame of the one or more first image frames that has the level of similarity with the second image frame that meets the threshold condition, designating the first image frame as a candidate image frame; 
 combining designated candidate image frames to generate a combined image frame; and 
 combining the combined image frame with the second image frame to generate the final image frame. 
 
     
     
       8. The system of  claim 1 , wherein the at least one processor circuit is configured to capture the second image frame with a first exposure value of the camera that causes the second image frame to be either underexposed or overexposed based on an amount of light measured by the image sensor. 
     
     
       9. The system of  claim 1 , wherein the image sensor is configured to capture the one or more first image frames with a second exposure value that is different than the first exposure value. 
     
     
       10. A method, comprising:
 capturing one or more first image frames during a preview mode of a camera; 
 detecting activation of a shutter button of the camera; 
 responsive to detecting activation of the shutter button, capturing a second image frame; 
 determining that a level of similarity between a first image frame of the one or more first image frames and the second image frame meets a threshold condition; 
 responsive to said determining, combining the first image frame with the second image frame to generate a final image frame; and 
 storing the final image frame in a memory device associated with the camera. 
 
     
     
       11. The method of  claim 10 , wherein the camera is included in a mobile computing device. 
     
     
       12. The method of  claim 10 , wherein the camera a dedicated camera device. 
     
     
       13. The method of  claim 10 , wherein the one or more first image frames are stored in a ring buffer. 
     
     
       14. The method of  claim 13 , further comprising:
 during the preview mode of the camera:
 comparing a third image frame of the one or more first image frames stored in the ring buffer to a fourth image frame of the one or more first image frames stored in the ring buffer, the fourth image frame being stored in the ring buffer at a time prior to when the third image frame is stored in the ring buffer; 
 determining that said comparing does not result in a level of similarity that meets the threshold condition; and 
 responsive to determining that said comparing does not result in a level of similarity that meets the threshold condition, discarding the fourth image frame. 
 
 
     
     
       15. The method of  claim 14 , wherein the third image frame is the first image frame. 
     
     
       16. The method of  claim 13 , wherein determining that the level of similarity between a first image frame of the one or more first image frames and the second image frame meets the threshold condition comprises:
 for each first image frame of the one or more first image frames that has the level of similarity with the second image frame that meets the threshold condition, designating the first image frame as a candidate image frame; 
 combining designated candidate image frames to generate a combined image frame; and 
 combining the combined image frame with the second image frame to generate the final image frame. 
 
     
     
       17. The method of  claim 10 , wherein capturing the second image frame comprises:
 capturing the second image frame with a first exposure value of the camera that causes the second image frame to be either underexposed or overexposed based on an amount of light measured by an image sensor of the camera. 
 
     
     
       18. The method of  claim 10 , wherein capturing the one or more first image frames comprises:
 capturing the one or more first image frames with a second exposure value that is different than the first exposure value. 
 
     
     
       19. A computer-readable storage device having program instructions recorded thereon that, when executed by at least one processor, perform a method comprising: capturing one or more first image frames during a preview mode of a camera; detecting activation of a shutter button of the camera; responsive to detecting activation of the shutter button, capturing a second image frame; determining that a level of similarity between a first image frame of the one or more first image frames and the second image frame meets a threshold condition; responsive to said determining, combining the first image frame with the second image frame to generate a final image frame; and storing the final image frame in a memory device associated with the camera. 
     
     
       20. The computer-readable storage device of  claim 19 , wherein the camera is included in a mobile computing device.