Dynamic and adaptive image brightness enhancement

Abstract

A method includes obtaining, using at least one imaging sensor of an electronic device, a color image frame of a scene. The method also includes determining, using at least one processing device of the electronic device, that a visual quality of the color image frame falls outside of a visual quality threshold. The method further includes, in response to determining that the visual quality of the color image frame falls outside of the visual quality threshold, performing, using the at least one processing device, visual quality enhancement to the color image frame to generate a modified image frame. In addition, the method includes applying, using the at least one processing device, one or more passthrough transformations to the modified image frame to generate a transformed image frame.

What is claimed is: 
     
         1 . A method comprising:
 obtaining, using at least one imaging sensor of an electronic device, a color image frame of a scene;   determining, using at least one processing device of the electronic device, that a visual quality of the color image frame falls outside of a visual quality threshold;   in response to determining that the visual quality of the color image frame falls outside of the visual quality threshold, performing, using the at least one processing device, visual quality enhancement to the color image frame to generate a modified image frame; and   applying, using the at least one processing device, one or more passthrough transformations to the modified image frame to generate a transformed image frame.   
     
     
         2 . The method of  claim 1 , wherein determining that the visual quality of the color image frame falls outside of the visual quality threshold comprises:
 segmenting the color image frame into a plurality of regions;   obtaining regional visual quality parameters for each of the regions and global visual quality parameters for the color image frame, the regional visual quality parameters including a regional signal-to-noise ratio, a regional brightness, and a regional histogram, the global visual quality parameters including a global signal-to-noise ratio, a global brightness, and a global histogram;   identifying a regional visual quality threshold for each region based on the regional visual quality parameters associated with the region; and   identifying a global visual quality threshold for the color image frame based on the global visual quality parameters.   
     
     
         3 . The method of  claim 2 , further comprising:
 labeling one or more first regions to be enhanced and one or more second regions to not be enhanced;   wherein each first region has an associated regional visual quality parameter that falls outside of the regional visual quality threshold associated with the first region; and   wherein each second region has an associated regional visual quality parameter within the regional visual quality threshold associated with the second region.   
     
     
         4 . The method of  claim 2 , wherein performing the visual quality enhancement comprises at least one of:
 applying a local visual enhancement to one or more of the regions; or   applying a global visual enhancement to the color image frame.   
     
     
         5 . The method of  claim 4 , wherein applying the local visual enhancement comprises:
 selecting one or more visual enhancement algorithms for the one or more regions, the one or more visual enhancement algorithms including at least one of: a histogram processing, a contrast processing, or an exposure processing; and   applying the one or more visual enhancement algorithms to the one or more regions.   
     
     
         6 . The method of  claim 4 , wherein applying the global visual enhancement comprises:
 selecting one or more global visual enhancement algorithms for the color image frame, the one or more global visual enhancement algorithms including at least one of: a histogram processing, an exposure processing, a contrast processing, or a relighting; and   applying the one or more visual enhancement algorithms to the color image frame such that the brightness is more consistent throughout the color image frame.   
     
     
         7 . The method of  claim 1 , further comprising:
 converting a color format of the color image frame;   extracting a luminance component from the converted color format of the color image frame; and   determining brightness of the color image frame using the luminance component.   
     
     
         8 . The method of  claim 1 , further comprising:
 rendering a final image frame for display based on the transformed image frame.   
     
     
         9 . An apparatus comprising:
 at least one processing device configured to:
 obtain a color image frame of a scene; 
 determine that a visual quality of the color image frame falls outside of a visual quality threshold; 
 in response to determining that the visual quality of the color image frame falls outside of the visual quality threshold, perform visual quality enhancement to the color image frame to generate a modified image frame; and 
 apply one or more passthrough transformations to the modified image frame to generate a transformed image frame. 
   
     
     
         10 . The apparatus of  claim 9 , wherein, to determine that the visual quality of the color image frame falls outside of the visual quality threshold, the at least one processing device is configured to:
 segment the color image frame into a plurality of regions;   obtain regional visual quality parameters for each of the regions and global visual quality parameters for the color image frame, the regional visual quality parameters including a regional signal-to-noise ratio, a regional brightness, and a regional histogram, the global visual quality parameters including a global signal-to-noise ratio, a global brightness, and a global histogram;   identify a regional visual quality threshold for each region based on the regional visual quality parameters associated with the region; and   identify a global visual quality threshold for the color image frame based on the global visual quality parameters.   
     
     
         11 . The apparatus of  claim 10 , wherein:
 the at least one processing device is further configured to label one or more first regions to be enhanced and one or more second regions to not be enhanced;   each first region has an associated regional visual quality parameter that falls outside of the regional visual quality threshold associated with the first region; and   each second region has an associated regional visual quality parameter within the regional visual quality threshold associated with the second region.   
     
     
         12 . The apparatus of  claim 10 , wherein, to perform the visual quality enhancement, the at least one processing device is configured to at least one of:
 apply a local visual enhancement to one or more of the regions; or   apply a global visual enhancement to the color image frame.   
     
     
         13 . The apparatus of  claim 12 , wherein, to apply the local visual enhancement, the at least one processing device is configured to:
 select one or more visual enhancement algorithms for the one or more regions, the one or more visual enhancement algorithms including at least one of: a histogram processing, a contrast processing, or an exposure processing; and   apply the one or more visual enhancement algorithms to the one or more regions.   
     
     
         14 . The apparatus of  claim 12 , wherein, to apply the global visual enhancement, the at least one processing device is configured to:
 select one or more global visual enhancement algorithms for the color image frame, the one or more global visual enhancement algorithms including at least one of: a histogram processing, an exposure processing, a contrast processing, or a relighting; and   apply the one or more visual enhancement algorithms to the color image frame such that the brightness is more consistent throughout the color image frame.   
     
     
         15 . The apparatus of  claim 9 , wherein the at least one processing device is further configured to:
 convert a color format of the color image frame;   extract a luminance component from the converted color format of the color image frame; and   determine brightness of the color image frame using the luminance component.   
     
     
         16 . A non-transitory machine readable medium containing instructions that when executed cause at least one processor of an electronic device to:
 obtain a color image frame of a scene;   determine that a visual quality of the color image frame falls outside of a visual quality threshold;   in response to determining that the visual quality of the color image frame falls outside of the visual quality threshold, perform visual quality enhancement to the color image frame to generate a modified image frame; and   apply one or more passthrough transformations to the modified image frame to generate a transformed image frame.   
     
     
         17 . The non-transitory machine readable medium of  claim 16 , wherein the instructions that when executed cause the at least one processor to determine that the visual quality of the color image frame falls outside of the visual quality threshold comprise instructions that when executed cause the at least one processor to:
 segment the color image frame into a plurality of regions;   obtain regional visual quality parameters for each of the regions and global visual quality parameters for the color image frame, the regional visual quality parameters including a regional signal-to-noise ratio, a regional brightness, and a regional histogram, the global visual quality parameters including a global signal-to-noise ratio, a global brightness, and a global histogram;   identify a regional visual quality threshold for each region based on the regional visual quality parameters associated with the region; and   identify a global visual quality threshold for the color image frame based on the global visual quality parameters.   
     
     
         18 . The non-transitory machine readable medium of  claim 17 , wherein:
 the instructions that when executed cause the at least one processor to determine that the visual quality of the color image frame falls outside of the visual quality threshold further comprise instructions that when executed cause the at least one processor to label one or more first regions to be enhanced and one or more second regions to not be enhanced;   each first region has an associated regional visual quality parameter that falls outside of the regional visual quality threshold associated with the first region; and   each second region has an associated regional visual quality parameter within the regional visual quality threshold associated with the second region.   
     
     
         19 . The non-transitory machine readable medium of  claim 17 , wherein the instructions that when executed cause the at least one processor to perform the visual quality enhancement comprise instructions that when executed cause the at least one processor to at least one of:
 apply a local visual enhancement to one or more of the regions; or   apply a global visual enhancement to the color image frame.   
     
     
         20 . The non-transitory machine readable medium of  claim 19 , wherein:
 the instructions that when executed cause the at least one processor to apply the local visual enhancement comprise instructions that when executed cause the at least one processor to:
 select one or more visual enhancement algorithms for the one or more regions, the one or more visual enhancement algorithms including at least one of: a histogram processing, a contrast processing, or an exposure processing; and 
 apply the one or more visual enhancement algorithms to the one or more regions; and 
   the instructions that when executed cause the at least one processor to apply the global visual enhancement comprise instructions that when executed cause the at least one processor to:
 select one or more global visual enhancement algorithms for the color image frame, the one or more global visual enhancement algorithms including at least one of: a histogram processing, an exposure processing, a contrast processing, or a relighting; and 
 apply the one or more visual enhancement algorithms to the color image frame such that the brightness is more consistent throughout the color image frame.