Method, system and program product for evaluating a data mining algorithm

Abstract

An improved solution for evaluating one or more data mining algorithms. A set of goals for the data mining algorithm(s) is obtained, and a weight can be assigned to each goal. Each data mining algorithm is applied to a dataset to generate a set of results. A performance value for each data mining algorithm can be calculated based on the weights and set of results. When multiple data mining algorithms are being evaluated, their respective performances can be compared using the respective sets of results and performance values.

1 . A method of evaluating a data mining algorithm, the method comprising: 
 obtaining a set of goals for the data mining algorithm;    assigning a weight to each goal in the set of goals;    applying the data mining algorithm to a dataset; and    calculating a performance value for the data mining algorithm based on the set of weights and a set of results for the applying step.    
   
   
       2 . The method of  claim 1 , wherein the assigning step includes: 
 identifying a set of error cases for each goal in the set of goals; and    assigning a weight to each error case in the set of error cases.    
   
   
       3 . The method of  claim 2 , wherein the assigning step includes: 
 obtaining an acceptability for an error case; and    calculating the weight based on the acceptability.    
   
   
       4 . The method of  claim 2 , wherein the calculating step includes: 
 determining an error rate for each error case based on the set of results; and    calculating an error vector for each error case based on the error rate and error weight for the error case.    
   
   
       5 . The method of  claim 4 , wherein the calculating step further includes summing the error vectors for the set of error cases to obtain the performance value.  
   
   
       6 . The method of  claim 1 , further comprising comparing the performance value to an acceptable performance value.  
   
   
       7 . A method of evaluating a set of data mining algorithms, the method comprising: 
 selecting the set of data mining algorithms;    obtaining a set of goals for the set of data mining algorithms;    assigning a weight to each goal in the set of goals;    applying each data mining algorithm to a dataset; and    calculating a performance value for each data mining algorithm based on the set of weights and a set of results for the applying step.    
   
   
       8 . The method of  claim 7 , wherein the selecting step is based on the set of goals.  
   
   
       9 . The method of  claim 7 , wherein the selecting step includes: 
 selecting a business taxonomy;    selecting a business problem based on the business taxonomy; and    selecting the set of data mining algorithms based on the business problem.    
   
   
       10 . The method of  claim 7 , further comprising ranking the set of data mining algorithms based on the performance values.  
   
   
       11 . The method of  claim 7 , wherein the assigning step includes: 
 identifying a set of error cases for each goal; and    assigning a weight to each error case in the set of error cases.    
   
   
       12 . The method of  claim 7 , wherein the set of data mining algorithms includes at least one data mining algorithm having a first set of parameter values and the at least one data mining algorithm having a second set of parameter values.  
   
   
       13 . The method of  claim 7 , further comprising: 
 selecting a data mining algorithm in the set of data mining algorithms; and    generating a data mining model based on the selected data mining algorithm.    
   
   
       14 . A system for evaluating a set of data mining algorithms having a set of goals, the system comprising: 
 an assignment system for assigning a weight to each goal in the set of goals;    an application system for applying each data mining algorithm to a dataset; and    a performance system for calculating a performance value for each data mining algorithm based on the weights assigned to the set of goals and a set of results for the applying step.    
   
   
       15 . The system of  claim 14 , further comprising a selection system for selecting the set of data mining algorithms.  
   
   
       16 . The system of  claim 14 , further comprising a ranking system for ranking the set of data mining algorithms based on the performance values.  
   
   
       17 . The system of  claim 14 , further comprising a summary system for displaying the performance values for at least some of the set of data mining algorithms to a user.  
   
   
       18 . The system of  claim 14 , further comprising a generation system for generating a data mining model based on a data mining algorithm selected from the set of data mining algorithms.  
   
   
       19 . The system of  claim 14 , wherein the application system applies the set of data mining algorithms in parallel.  
   
   
       20 . A program product stored on a recordable medium for evaluating a set of data mining algorithms having a set of goals, which when executed comprises: 
 program code for assigning a weight to each goal in the set of goals;    program code for applying each data mining algorithm to a dataset; and    program code for calculating a performance value for each data mining algorithm based on the weights assigned to the set of goals and a set of results for the applying step.    
   
   
       21 . The program product of  claim 20 , further comprising program code for selecting the set of data mining algorithms.  
   
   
       22 . The program product of  claim 20 , further comprising program code for ranking the set of data mining algorithms based on the performance values.