Predicting optimal values for parameters used in an operation of an image signal processor using machine learning

Abstract

A method of predicting optimal values for a plurality of parameters used in an operation of an image signal processor includes: inputting initial values for the plurality of parameters to a machine learning model having an input layer, corresponding to the plurality of parameters, and an output layer corresponding to a plurality of evaluation items extracted from a result image generated by the image signal processor; obtaining evaluation scores for the plurality of evaluation items using an output of the machine learning model; adjusting weights, applied to the plurality of parameters, based on the evaluation scores; and determining the optimal values using the adjusted weights.

1 . A method of training a machine learning model to predict optimal values for a plurality of parameters used in an operation of an image signal processor, comprising:
 capturing an image of a sample subject to obtain sample data;   generating a plurality of sets of sample values for the plurality of parameters;   emulating the image signal processor (ISP) processing the sample data according to each of the sets to generate a plurality of sample images;   evaluating each of the plurality of sample images for a plurality of evaluation items to generate respective sample scores; and   training the machine learning model to predict the optimal values using the sample values and the sample scores.   
     
     
         2 . The method of  claim 1 , wherein the plurality of parameters include at least two of a color, blurring, noise, a contrast ratio, a resolution, and a size of an image. 
     
     
         3 . The method of  claim 1 , wherein the plurality of evaluation items include at least two of a color, sharpness, noise, a resolution, a dynamic range, shading, and texture loss of an image. 
     
     
         4 . The method of  claim 1 , wherein the plurality of sets of the sample values include a first sample set and a second sample set for the plurality of parameters, and
 the plurality of sample images include a first sample image, corresponding to the first sample set, and a second sample image corresponding to the second sample set.   
     
     
         5 . The method of  claim 4 , wherein the sample scores comprise a first sample score set obtained from the first sample image, and a second sample score set obtained from the second sample image. 
     
     
         6 . The method of  claim 1 , the training of the machine learning model comprising:
 inputting initial values, for the plurality of parameters, to the machine learning model to obtain evaluation scores for the plurality of evaluation items; and   adjusting weights, applied to the plurality of parameters, such that the evaluation scores satisfy predetermined reference conditions.   
     
     
         7 . The method of  claim 6 , further comprising: inputting raw data to the image signal processor, having the plurality of parameters to which the weights are applied, to generate a result image when an image sensor captures a subject to generate the raw data. 
     
     
         8 . The method of  claim 6 , wherein the machine learning model is implemented as an artificial neural network. 
     
     
         9 . The method of  claim 6 , wherein the weights and the plurality of parameters are connected in a partially connected manner. 
     
     
         10 . The method of  claim 1 , wherein the sample subject includes a plurality of different subjects. 
     
     
         11 . A method of predicting optimal values for a plurality of parameters used in an operation of an image signal processor, comprising:
 inputting initial values for the plurality of parameters to a machine learning model including an input layer having a plurality of input nodes, corresponding to the plurality of parameters, and an output layer having a plurality of output nodes, corresponding to a plurality of evaluation items extracted from a result image generated by the image signal processor;   obtaining evaluation scores for the plurality of evaluation items using an output of the machine learning model;   adjusting weights, applied to the plurality of parameters, based on the evaluation scores; and   determining the optimal values using the adjusted weights.   
     
     
         12 . The method of  claim 11 , wherein at least some of the weights are adjusted by a user of a device in which the image signal processor is mounted. 
     
     
         13 . The method of  claim 11 , wherein the evaluation scores are obtained while adjusting the weights, and the adjustment of the weights completes when each of the evaluation scores satisfies predetermined reference conditions. 
     
     
         14 . The method of  claim 11 , wherein the evaluation scores are obtained while adjusting the weights a predetermined number of times. 
     
     
         15 . The method of  claim 11 , further comprising:
 tuning the image signal processor using the optimal values.   
     
     
         16 . The method of  claim 11 , wherein at least some of the weights have different values depending on a subject captured by the image sensor. 
     
     
         17 . The method of  claim 11 , further comprising:
 generating a plurality of sets of sample values for the plurality of parameters;   emulating the image signal processor (ISP) processing the sample data according to each of the sets to generate a plurality of sample images;   evaluating each of the plurality of sample images for a plurality of evaluation items to generate respective sample scores; and   training the machine learning model using the sample values and the sample scores.   
     
     
         18 . An electronic device comprising:
 an image signal processor configured to process raw data, output by an image sensor, depending on a plurality of parameters to generate a result image; and   a parameter optimization module including a machine learning model, receiving sample values for the plurality of parameters and outputting a plurality of sample scores indicating quality of sample images, the sample images being generated by the image signal processor processing the raw data based on the sample values, the parameter optimization module being configured to determine weights, respectively applied to the plurality of parameters, using the machine learning model,   wherein the image signal processor applies the weights to the plurality of parameters to generate a plurality of weighted parameters and generates the result image by processing the raw data using the weighted parameters.   
     
     
         19 . The electronic device of  claim 18 , wherein the image signal processor and the parameter optimization module are mounted on a single integrated circuit chip. 
     
     
         20 . The electronic device of  claim 18 , wherein the image signal processor and the image sensor are mounted on a single integrated circuit chip. 
     
     
         21 - 22 . (canceled)