Method and system for automatically colorizing night-vision images

Abstract

A method as well as a system are provided for generating a first color image from a first night-vision image. First, predicted chrominance values are determined by using a prediction function based on luminance values of the first night-vision image. Then the first color image is generated by combining the luminance values of the first night-vision image with the predicted chrominance values.

1 . A method for generating a first color image from a first night-vision image, the method comprising the steps of:
 determining, by using a prediction function, predicted chrominance values based on luminance values of the first night-vision image; and   generating the first color image by combining the luminance values of the first night-vision image with the predicted chrominance values.   
     
     
         2 . The method according to  claim 1 , further comprising the step of determining the prediction function which maps luminance values of night-vision images to predicted chrominance values, wherein the predicted chrominance values are similar to chrominance values of color images corresponding to the night-vision images. 
     
     
         3 . The method according to  claim 2 , wherein determining the prediction function which maps the luminance values of night-vision images to the predicted chrominance values of the corresponding color images comprises the following steps:
 obtaining several pairs of training images, each pair of training images comprising a night-vision image and a color image, wherein the night-vision image and the color image depict an overlapping and/or substantially identical region of interest;   determining the prediction function by training a machine learning model on the training images to predict the chrominance of the color images from the luminance of the night-vision images.   
     
     
         4 . The method according to  claim 1 , wherein the prediction function determines the predicted chrominance values based on the luminance values of the first night-vision image without taking into account captured color and/or chrominance values except for color and/or chrominance values of training images used for determining the prediction function. 
     
     
         5 . The method according to  claim 3  wherein the first night-vision image and/or the night-vision images of the several pairs of training images are infrared or near infrared images obtained while using an active source of infrared illumination. 
     
     
         6 . The method according to  claim 3 , wherein each pair of training images is obtained by simultaneously, using daylight or artificial light, capturing a night-vision image and a color image containing substantially identical regions of interest. 
     
     
         7 . The method according to  claim 3 , further comprising the step of adapting the prediction function when new pairs of training images are obtained. 
     
     
         8 . The method according to  claim 3 , wherein the machine learning method used for determining the prediction function is based on neural networks, preferably convolutional neural networks. 
     
     
         9 . The method according to  claim 3 , wherein training the prediction function comprises identifying feature maps based on characteristic shapes and shading of the pairs of training images and wherein the predicted chrominance values are determined by the prediction function based on the first night-vision image taking into account said feature maps. 
     
     
         10 . A system for automatically generating a first color image from a first night-vision image, the system comprising:
 at least one camera configured for capturing the first night-vision image;   a processing unit configured for determining, by using a prediction function which maps luminance values of night-vision images to chrominance values of corresponding color images, predicted chrominance values for the luminance values of the first night-vision image; and for generating the first color image by combining the luminance values of the first night-vision image with the predicted chrominance values; and   a display for displaying the first color image to a user.   
     
     
         11 . The system according to  claim 10 , wherein the system further comprises another processing unit which is configured for:
 obtaining several pairs of training images, each pair of training images comprising a night-vision image and a corresponding color image, wherein the night-vision image and the color image depict overlapping and/or substantially identical regions of interest;   determining the prediction function by training a machine learning method to predict the chrominance of the color images from the luminance of the night-vision images.   
     
     
         12 . The system according to  claim 11 , wherein the at least one camera is an infrared camera and the system further comprises:
 at least one infrared illumination source, wherein the at least one camera is configured for capturing the infrared or near infrared images while the infrared illumination source is active.   
     
     
         13 . The system according to  claim 11 , wherein the other processing unit is further configured to adapt the prediction function when new pairs of training images are obtained. 
     
     
         14 . The system according to  claim 10  wherein the camera is a stationary camera. 
     
     
         15 . The system according to  claim 10  wherein the camera is a non-stationary camera.