Using machine learning to filter monte carlo noise from images

Abstract

A method of producing noise-free images is disclosed. The method includes using machine learning incorporating a filter to output filter parameters using the training images. The machine learning may include training a neural network. The filter parameters are applied to Monte Carlo rendered training images that have noise to generate noise-free images. The training may include determining, computing and extracting features of the training images; computing filter parameters; applying an error metric; and applying backpropgation. The neural network may be a multilayer perceptron. The machine learning model is applied to new noisy Monte Carlo rendered images to create noise-free images. This may include applying the filter to the noisy Monte Carlo rendered images using the filter parameters to create the noise-free images.

It is claimed: 
     
         1 . A method of producing a noise-free image, the method comprising:
 obtaining training images;   using machine learning incorporating a filter on the training images to output filter parameters;   receiving a Monte Carlo rendered image that has noise;   executing the filter on the noisy image using the filter parameters to generate the noise-free image.   
     
     
         2 . The method of  claim 1  wherein the training images include both ground truth training images and noisy training images. 
     
     
         3 . The method of  claim 1  wherein the using machine learning is training a neural network. 
     
     
         4 . The method of  claim 3  wherein the neural network is a multilayer perceptron. 
     
     
         5 . The method of  claim 3  wherein the training the neural network includes:
 extracting, determining and/or computing features from the training images; 
 computing testing filter parameters using the machine learning model including applying the filter using the features to create a temporary image; 
 applying an error metric to the temporary image; 
 correcting the machine learning model based on the error metric including updating the testing filter parameters; 
 repeating the computing, the applying and the correcting to determine final filter parameters. 
 
     
     
         6 . The method of  claim 5  wherein the extracting, determining and/or computing features includes:
 determining primary features of the training images; 
 extracting and/or computing secondary features of the training images using the primary features. 
 
     
     
         7 . The method of  claim 6  wherein the machine learning is a neural network of the secondary features. 
     
     
         8 . The method of  claim 6  wherein the primary features include some selected from the group including: positions, colors, world positions, visibility, shading normals, texture values. 
     
     
         9 . The method of  claim 6  wherein the secondary features include some selected from the group including: variances and noise approximation in local regions, mean of primary features at various block sizes, standard deviation of the primary features at various block sizes, gradients of primary features, mean deviation of the primary features, median absolute deviation (MAD) of primary features, sampling rate. 
     
     
         10 . The method of  claim 3  further including applying backpropagation on the neural network. 
     
     
         11 . The method of  claim 5  wherein the error metric is a modified relative mean squared error function. 
     
     
         12 . The method of  claim 5  wherein the error metric is a perceptual metric such as a structural similarity index (SSIM). 
     
     
         13 . The method of  claim 1  wherein executing the filter includes one selected from the group including a Gaussian filter, a cross-bilateral filter, and a cross non-local means filter. 
     
     
         14 . The method of  claim 1  wherein using machine learning includes:
 applying an error metric to measure the distance between filtered images and ground truth images; 
 applying an optimization strategy to minimize an energy function on results of the error metric. 
 
     
     
         15 . The method of  claim 14  wherein the energy function computes errors in a multilayer perceptron. 
     
     
         16 . The method of  claim 14  wherein the error metric is a modified relative mean squared error (RelMSE) metric. 
     
     
         17 . The method of  claim 14  wherein the optimization strategy is backpropagation. 
     
     
         18 . A method of producing a noise-free Monte Carlo rendered image, the method comprising:
 training a machine learning model on a plurality of Monte Carlo-rendered training images to learn how to output noise-free images;   receiving a new Monte Carlo rendered noisy image;   executing the trained machine learning model on the new Monte Carlo rendered noisy image to generate a noise-free result.   
     
     
         19 . The method of  claim 18 ,
 wherein the training includes placing a filter after the machine learning model and the machine learning model outputs parameters for the filter   wherein the executing including applying the filter to denoise the new Monte Carlo rendered noisy image.   
     
     
         20 . The method of  claim 19 , wherein during the training the machine learning model is trained to output optimal filter parameters for the training images such that when the new Monte Carlo noisy image is received, the machine learning model outputs the filter parameters to remove noise from the new image. 
     
     
         21 . The method of  claim 18  wherein the machine learning model is a neural network.