Automated early anomaly detection in a continuous learning model

Abstract

Embodiments of the present invention provide a method, system and computer program product for automated early anomaly detection in a continuous learning model. In an embodiment of the invention, a method includes training a continuous learning model with a training data set of different records and a known target class for each of the different records, deploying the model, and monitoring performance of the model. The method further includes prior to receiving a complete feedback data set for the model, computing a metric in the model based upon unseen records in the model that had not been present in the training data set, determining poor quality of the model for a metric computed to exceed a threshold value and displaying a recommendation in the host server to retrain the model responsive to the determination of poor quality of the model.

We claim: 
     
         1 . A method for automated early anomaly detection in a continuous learning model comprising:
 training a continuous learning model with a training data set of different records and a known target class for each of the different records;   deploying the continuous learning model in memory of a host server;   monitoring performance of the continuous learning model; and,   prior to receiving a complete feedback data set for the continuous learning model, computing a metric in the continuous learning model based upon unseen records in the continuous learning model that had not been present in the training data set, determining poor quality of the continuous learning model for a metric computed to exceed a threshold value and displaying a recommendation in the host server to retrain the continuous learning model responsive to the determination of poor quality of the continuous learning model.   
     
     
         2 . The method of  claim 1 , wherein the metric is computed based upon a number of unseen records in the continuous learning model relative to a total number of records in the continuous learning model. 
     
     
         3 . The method of  claim 2 , wherein the metric is computed based upon a number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model. 
     
     
         4 . The method of  claim 3 , wherein the metric is a combination of the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model and the number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model. 
     
     
         5 . The method of  claim 4 , wherein each of the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model, and the number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model are weighted within the combination. 
     
     
         6 . A data processing system configured for automated early anomaly detection in a continuous learning model, the system comprising:
 a host computing system comprising memory and at least one processor;   fixed storage coupled to the host computing system; and,   an automated early anomaly detection module comprising computer program instructions executing in the memory of the host computing system that upon execution are adapted to perform:   training a continuous learning model with a training data set of different records and a known target class for each of the different records;   deploying the continuous learning model in the memory of the host computing system;   monitoring performance of the continuous learning model; and,   prior to receiving a complete feedback data set for the continuous learning model, computing a metric in the continuous learning model based upon unseen records in the continuous learning model that had not been present in the training data set, determining poor quality of the continuous learning model for a metric computed to exceed a threshold value and displaying a recommendation in the host server to retrain the continuous learning model responsive to the determination of poor quality of the continuous learning model.   
     
     
         7 . The system of  claim 6 , wherein the metric is computed based upon a number of unseen records in the continuous learning model relative to a total number of records in the continuous learning model. 
     
     
         8 . The system of  claim 7 , wherein the metric is computed based upon a number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model. 
     
     
         9 . The system of  claim 8 , wherein the metric is a combination of the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model and the number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model. 
     
     
         10 . The system of  claim 9 , wherein each of the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model, and the number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model are weighted within the combination. 
     
     
         11 . A computer program product for automated early anomaly detection in a continuous learning model, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a device to cause the device to perform a method comprising:
 training a continuous learning model with a training data set of different records and a known target class for each of the different records;   deploying the continuous learning model in memory of a host server;   monitoring performance of the continuous learning model; and,   prior to receiving a complete feedback data set for the continuous learning model, computing a metric in the continuous learning model based upon unseen records in the continuous learning model that had not been present in the training data set, determining poor quality of the continuous learning model for a metric computed to exceed a threshold value and displaying a recommendation in the host server to retrain the continuous learning model responsive to the determination of poor quality of the continuous learning model.   
     
     
         12 . The computer program product of  claim 11 , wherein the metric is computed based upon a number of unseen records in the continuous learning model relative to a total number of records in the continuous learning model. 
     
     
         13 . The computer program product of  claim 12 , wherein the metric is computed based upon a number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model. 
     
     
         14 . The computer program product of  claim 13 , wherein the metric is a combination of the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model and the number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model. 
     
     
         15 . The computer program product of  claim 14 , wherein each of the number of unseen records in the continuous learning model relative to the total number of records in the continuous learning model, and the number of unique ones of the unseen records in the continuous learning model relative to the number of unseen records in the continuous learning model are weighted within the combination.