Computer-readable recording medium storing machine learning program, apparatus, and method

Abstract

A recording medium stores a machine learning program for causing a computer to execute processing including: acquiring, in deep learning of a model that includes layers, information that indicates a learning status for each iterative processing of learning processing; determining progress of learning based on the information that indicates the learning status; skipping a part of learning processing of each layer included in a first layer group from a input layer to a specific layer and in which the progress of the learning satisfies a condition; and restarting the part of the learning processing skipped when the part of the learning processing is skipped and a change amount of an evaluation value, which is based on the information that indicates the learning status, of any of layers included in a second layer group from a next layer of the specific layer to an output layer exceeds a threshold range.

What is claimed is: 
     
         1 . A non-transitory computer-readable recording medium storing a machine learning program for causing a computer to execute processing comprising:
 acquiring, in deep learning of a model that includes a plurality of layers that includes an input layer and an output layer, information that indicates a learning status for each iterative processing of learning processing;   determining progress of learning of each layer on the basis of the information that indicates the learning status;   skipping a part of learning processing of each layer which is included in a first layer group from the input layer to a specific layer and in which the progress of the learning satisfies a predetermined condition; and   restarting the part of the learning processing skipped in each layer included in the first layer group in a case where the part of the learning processing of each layer included in the first layer group is skipped and a change amount of an evaluation value, which is based on the information that indicates the learning status, of any of layers included in a second layer group from a next layer of the specific layer, which is close to a side of the output layer, to the output layer exceeds a predetermined threshold range.   
     
     
         2 . The non-transitory computer-readable recording medium storing the machine learning program according to  claim 1 , wherein
 the learning processing includes first processing of calculating an error between an output value output from the output layer by inputting training data from the input layer and a correct answer to the training data, second processing of backward-propagating information regarding the error from the output layer to the input layer and calculating an error gradient for a weight between layers, and third processing of updating the weight between the layers by using the calculated error gradient, and   in a case where the part of the learning processing is skipped, the second processing and the third processing are skipped.   
     
     
         3 . The non-transitory computer-readable recording medium storing the machine learning program according to  claim 2 , wherein, in a case where the learning processing is processed by a plurality of arithmetic units, the error gradients calculated by executing the first processing and the second processing by using a different type of training data in each of the plurality of arithmetic units are integrated to obtain an error gradient used in the third processing. 
     
     
         4 . The non-transitory computer-readable recording medium storing the machine learning program according to  claim 1 , wherein the evaluation value is a value represented by using a weight between layers, an error gradient, or a momentum or any combination of the weight between layers, the error gradient, or the momentum. 
     
     
         5 . The non-transitory computer-readable recording medium storing the machine learning program according to  claim 4 , wherein an inner product of the error gradient and the momentum is used as the evaluation value. 
     
     
         6 . The non-transitory computer-readable recording medium storing the machine learning program according to  claim 1 , wherein
 the processing of acquiring the information that indicates the learning status includes acquiring the information that indicates the learning status for every one iteration, which is a minimum unit of iterative processing of the learning, and   the change amount of the evaluation value is a change amount between an evaluation value based on the information that indicates the learning status acquired in a current iteration and an evaluation value based on the information that indicates the learning status acquired in a preceding iteration, or a change amount between a statistical value of evaluation values based on the information that indicates the learning status, which are acquired in a predetermined number of iterations in a first period that includes the current iteration, and a statistical value of evaluation values based on the information that indicates the learning status, which are acquired in a predetermined number of iterations in a second period before the first period.   
     
     
         7 . A information processing apparatus comprising:
 a memory; and   a processor coupled to the memory and configured to:   acquire, in deep learning of a model that includes a plurality of layers that includes an input layer and an output layer, information that indicates a learning status for each iterative processing of learning processing;   determine progress of learning of each layer on the basis of the information that indicates the learning status;   skip a part of learning processing of each layer which is included in a first layer group from the input layer to a specific layer and in which the progress of the learning satisfies a predetermined condition; and   restart the part of the learning processing skipped in each layer included in the first layer group in a case where the part of the learning processing of each layer included in the first layer group is skipped and a change amount of an evaluation value, which is based on the information that indicates the learning status, of any of layers included in a second layer group from a next layer of the specific layer, which is close to a side of the output layer, to the output layer exceeds a predetermined threshold range.   
     
     
         8 . The information processing apparatus according to  claim 7 , wherein
 the learning processing includes first processing of calculating an error between an output value output from the output layer by inputting training data from the input layer and a correct answer to the training data, second processing of backward-propagating information regarding the error from the output layer to the input layer and calculating an error gradient for a weight between layers, and third processing of updating the weight between the layers by using the calculated error gradient, and   in a case where the part of the learning processing is skipped, the second processing and the third processing are skipped.   
     
     
         9 . The information processing apparatus according to  claim 8 , wherein, in a case where the learning processing is processed by a plurality of arithmetic units, the error gradients calculated by executing the first processing and the second processing by using a different type of training data in each of the plurality of arithmetic units are integrated to obtain an error gradient used in the third processing. 
     
     
         10 . The information processing apparatus according to  claim 7 , wherein the evaluation value is a value represented by using a weight between layers, an error gradient, or a momentum or any combination of the weight between layers, the error gradient, or the momentum. 
     
     
         11 . The information processing apparatus according to  claim 10 , wherein an inner product of the error gradient and the momentum is used as the evaluation value. 
     
     
         12 . The information processing apparatus according to  claim 7 , wherein
 the processing of acquiring the information that indicates the learning status includes acquiring the information that indicates the learning status for every one iteration, which is a minimum unit of iterative processing of the learning, and   the change amount of the evaluation value is a change amount between an evaluation value based on the information that indicates the learning status acquired in a current iteration and an evaluation value based on the information that indicates the learning status acquired in a preceding iteration, or a change amount between a statistical value of evaluation values based on the information that indicates the learning status, which are acquired in a predetermined number of iterations in a first period that includes the current iteration, and a statistical value of evaluation values based on the information that indicates the learning status, which are acquired in a predetermined number of iterations in a second period before the first period.   
     
     
         13 . A machine learning method comprising:
 acquiring, by a computer, in deep learning of a model that includes a plurality of layers that includes an input layer and an output layer, information that indicates a learning status for each iterative processing of learning processing;   determining progress of learning of each layer on the basis of the information that indicates the learning status;   skipping a part of learning processing of each layer which is included in a first layer group from the input layer to a specific layer and in which the progress of the learning satisfies a predetermined condition; and   restarting the part of the learning processing skipped in each layer included in the first layer group in a case where the part of the learning processing of each layer included in the first layer group is skipped and a change amount of an evaluation value, which is based on the information that indicates the learning status, of any of layers included in a second layer group from a next layer of the specific layer, which is close to a side of the output layer, to the output layer exceeds a predetermined threshold range.   
     
     
         14 . The machine learning method according to  claim 13 , wherein
 the learning processing includes first processing of calculating an error between an output value output from the output layer by inputting training data from the input layer and a correct answer to the training data, second processing of backward-propagating information regarding the error from the output layer to the input layer and calculating an error gradient for a weight between layers, and third processing of updating the weight between the layers by using the calculated error gradient, and   in a case where the part of the learning processing is skipped, the second processing and the third processing are skipped.   
     
     
         15 . The machine learning method according to  claim 14 , wherein, in a case where the learning processing is processed by a plurality of arithmetic units, the error gradients calculated by executing the first processing and the second processing by using a different type of training data in each of the plurality of arithmetic units are integrated to obtain an error gradient used in the third processing. 
     
     
         16 . The machine learning method according to  claim 13 , wherein the evaluation value is a value represented by using a weight between layers, an error gradient, or a momentum or any combination of the weight between layers, the error gradient, or the momentum. 
     
     
         17 . The machine learning method according to  claim 16 , wherein an inner product of the error gradient and the momentum is used as the evaluation value. 
     
     
         18 . The machine learning method according to  claim 13 , wherein
 the processing of acquiring the information that indicates the learning status includes acquiring the information that indicates the learning status for every one iteration, which is a minimum unit of iterative processing of the learning, and   the change amount of the evaluation value is a change amount between an evaluation value based on the information that indicates the learning status acquired in a current iteration and an evaluation value based on the information that indicates the learning status acquired in a preceding iteration, or a change amount between a statistical value of evaluation values based on the information that indicates the learning status, which are acquired in a predetermined number of iterations in a first period that includes the current iteration, and a statistical value of evaluation values based on the information that indicates the learning status, which are acquired in a predetermined number of iterations in a second period before the first period.