System and method for mining model accuracy display

Abstract

Systems and methods are provided for producing displays of the accuracy of data mining or statistical models that produce associative predictions. For all cases in a testing data set, the model makes predictions and provides associated probabilities. The cases are sorted by their probability of making accurate predictions and a graph is made of the accuracy of the model over various subsets containing the highest probability cases as evaluated by the model. Where a number of probabilities are presented for the predictions in a basket of predictions, those probabilities are combined to yield a probability score for the entire basket. Additionally, the accuracy of a model over different basket sizes may be graphed. The accuracy graph may also be produced for any models making a prediction, by graphing the probability of making accurate predictions and a graph made of the accuracy of the model over various subsets of the data containing the highest probability cases.

1 . A method for accuracy display for a model, where, for each case in a testing data set containing one or more associated data items, said model predicts a basket containing a number of predicted data items associated with said case comprising: 
 producing a visual representation of a comprehensive association prediction evaluation on a graph of at least two dimensions, said comprehensive association prediction evaluation comprising at least two points where for each of said data points: a second coordinate of said data point corresponds to the accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point.    
   
   
       2 . A method according to  claim 1 , wherein said accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point corresponds to the number of cases for which said determination determines that said one of said associated data items removed from said case is contained in said basket predicted by said model.  
   
   
       3 . A method according to  claim 1 , wherein said accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point corresponds to the proportion of cases from among said testing data set for which said determination determines that said one of said associated data items removed from said case is contained in said basket predicted by said model.  
   
   
       4 . A method according to  claim 1 , where at a visual representation of a comprehensive association prediction evaluation for at least one other model is displayed on said graph.  
   
   
       5 . A modulated data signal carrying computer executable instructions for performing the method of  claim 1 .  
   
   
       6 . A computer-readable medium comprising computer-executable modules having computer-executable instructions for accuracy display for a model, where, for each case in a testing data set containing one or more associated data items, said model predicts a basket containing a number of predicted data items associated with said case, said modules comprising: 
 a module for producing a visual representation of a comprehensive association prediction evaluation on a graph of at least two dimensions, said comprehensive association prediction evaluation comprising at least two points where for each of said data points: a second coordinate of said data point corresponds to the accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point.    
   
   
       7 . A computer-readable medium according to  claim 6 , wherein said accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point corresponds to the number of cases for which said determination determines that said one of said associated data items removed from said case is contained in said basket predicted by said model.  
   
   
       8 . A computer-readable medium according to  claim 6 , wherein said accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point corresponds to the proportion of cases from among said testing data set for which said determination determines that said one of said associated data items removed from said case is contained in said basket predicted by said model.  
   
   
       9 . A computer-readable medium according to  claim 6 , where at a visual representation of a comprehensive association prediction evaluation for at least one other model is displayed on said graph.  
   
   
       10 . A computer device for accuracy display for a model, where, for each case in a testing data set containing one or more associated data items, said model predicts a basket containing a number of predicted data items associated with said case, comprising: 
 means producing a visual representation of a comprehensive association prediction evaluation on a graph of at least two dimensions, said comprehensive association prediction evaluation comprising at least two points where for each of said data points: a second coordinate of said data point corresponds to the accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point.    
   
   
       11 . A computer device according to  claim 10 , wherein said accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point corresponds to the number of cases for which said determination determines that said one of said associated data items removed from said case is contained in said basket predicted by said model.  
   
   
       12 . A computer device according to  claim 10 , wherein said accuracy of said model when predicting a basket containing a number of items corresponding to a first coordinate of said data point corresponds to the proportion of cases from among said testing data set for which said determination determines that said one of said associated data items removed from said case is contained in said basket predicted by said model.  
   
   
       13 . A computer device according to  claim 10 , where at a visual representation of a comprehensive association prediction evaluation for at least one other model is displayed on said graph.