Apparatus and method for managing machine learning

Abstract

A machine learning management apparatus identifies a maximum prediction performance score amongst a plurality of prediction performance scores corresponding to a plurality of models generated by executing each of a plurality of machine learning algorithms. As for a first machine learning algorithm having generated a model corresponding to the maximum prediction performance score, the machine learning management apparatus determines a first training dataset size to be used when the first machine learning algorithm is executed next time based on the maximum prediction performance score, first estimated prediction performance scores, and first estimated runtimes. As for a second machine learning algorithm different from the first machine learning algorithm, the machine learning management apparatus determines a second training dataset size to be used when the second machine learning algorithm is executed next time based on the maximum prediction performance score, second estimated prediction performance scores, and second estimated runtimes.

What is claimed is: 
     
         1 . A non-transitory computer-readable storage medium storing a computer program that causes a computer to perform a procedure comprising:
 identifying a maximum prediction performance score amongst a plurality of prediction performance scores corresponding to a plurality of models generated by executing each of a plurality of machine learning algorithms using one or more training dataset sizes;   calculating, for a first machine learning algorithm having generated a model corresponding to the maximum prediction performance score amongst the plurality of machine learning algorithms, based on execution results obtained by executing the first machine learning algorithm using the one or more training dataset sizes, first estimated prediction performance scores and first estimated runtimes for a case of executing the first machine learning algorithm using each of two or more training dataset sizes different from the one or more training dataset sizes, and determining, based on the maximum prediction performance score, the first estimated prediction performance scores, and the first estimated runtimes, a first training dataset size to be used when the first machine learning algorithm is executed next time; and   calculating, for a second machine learning algorithm different from the first machine learning algorithm amongst the plurality of machine learning algorithms, based on execution results obtained by executing the second machine learning algorithm using the one or more training dataset sizes, second estimated prediction performance scores and second estimated runtimes for a case of executing the second machine learning algorithm using each of two or more training dataset sizes different from the one or more training dataset sizes, and determining, based on the maximum prediction performance score, the second estimated prediction performance scores, and the second estimated runtimes, a second training dataset size to be used when the second machine learning algorithm is executed next time.   
     
     
         2 . The non-transitory computer-readable storage medium according to  claim 1 , wherein:
 the determining the first training dataset size includes calculating, for each of the two or more training dataset sizes, based on the maximum prediction performance score, the first estimated prediction performance scores, and the first estimated runtimes, a first increase rate indicating an increment in the maximum prediction performance score per unit time, and determining the first training dataset size based on calculated first increase rates, and   the determining the second training dataset size includes calculating, for each of the two or more training dataset sizes, based on the maximum prediction performance score, the second estimated prediction performance scores, and the second estimated runtimes, a second increase rate indicating an increment in the maximum prediction performance score per unit time, and determining the second training dataset size based on calculated second increase rates.   
     
     
         3 . The non-transitory computer-readable storage medium according to  claim 2 , wherein:
 the determining the first training dataset size includes setting, when a maximum first increase rate amongst the calculated first increase rates is higher than a maximum second increase rate amongst the calculated second increase rates, the first training dataset size larger than a training dataset size associated with the maximum first increase rate.   
     
     
         4 . The non-transitory computer-readable storage medium according to  claim 2 , wherein:
 the determining the second training dataset size includes setting, when the second estimated prediction performance scores and the second estimated runtimes satisfy a predetermined condition, the second training dataset size smaller than a training dataset size associated with a maximum second increase rate amongst the calculated second increase rates.   
     
     
         5 . A machine learning management apparatus comprising:
 a memory configured to store information on a plurality of prediction performance scores corresponding to a plurality of models generated by executing each of a plurality of machine learning algorithms using one or more training dataset sizes; and   a processor configured to perform a procedure including:
 identifying a maximum prediction performance score amongst the prediction performance scores, 
 calculating, for a first machine learning algorithm having generated a model corresponding to the maximum prediction performance score amongst the plurality of machine learning algorithms, based on execution results obtained by executing the first machine learning algorithm using the one or more training dataset sizes, first estimated prediction performance scores and first estimated runtimes for a case of executing the first machine learning algorithm using each of two or more training dataset sizes different from the one or more training dataset sizes, and determining, based on the maximum prediction performance score, the first estimated prediction performance scores, and the first estimated runtimes, a first training dataset size to be used when the first machine learning algorithm is executed next time, and 
 calculating, for a second machine learning algorithm different from the first machine learning algorithm amongst the plurality of machine learning algorithms, based on execution results obtained by executing the second machine learning algorithm using the one or more training dataset sizes, second estimated prediction performance scores and second estimated runtimes for a case of executing the second machine learning algorithm using each of two or more training dataset sizes different from the one or more training dataset sizes, and determining, based on the maximum prediction performance score, the second estimated prediction performance scores, and the second estimated runtimes, a second training dataset size to be used when the second machine learning algorithm is executed next time. 
   
     
     
         6 . A machine learning management method comprising:
 identifying, by a processor, a maximum prediction performance score amongst a plurality of prediction performance scores corresponding to a plurality of models generated by executing each of a plurality of machine learning algorithms using one or more training dataset sizes;   calculating, by the processor, for a first machine learning algorithm having generated a model corresponding to the maximum prediction performance score amongst the plurality of machine learning algorithms, based on execution results obtained by executing the first machine learning algorithm using the one or more training dataset sizes, first estimated prediction performance scores and first estimated runtimes for a case of executing the first machine learning algorithm using each of two or more training dataset sizes different from the one or more training dataset sizes, and determining, based on the maximum prediction performance score, the first estimated prediction performance scores, and the first estimated runtimes, a first training dataset size to be used when the first machine learning algorithm is executed next time; and   calculating, by the processor, for a second machine learning algorithm different from the first machine learning algorithm amongst the plurality of machine learning algorithms, based on execution results obtained by executing the second machine learning algorithm using the one or more training dataset sizes, second estimated prediction performance scores and second estimated runtimes for a case of executing the second machine learning algorithm using each of two or more training dataset sizes different from the one or more training dataset sizes, and determining, based on the maximum prediction performance score, the second estimated prediction performance scores, and the second estimated runtimes, a second training dataset size to be used when the second machine learning algorithm is executed next time.