Dynamic outlier bias reduction system and method

Abstract

A system and method is described herein for data filtering to reduce functional, and trend line outlier bias. Outliers are removed from the data set through an objective statistical method. Bias is determined based on absolute, relative error, or both. Error values are computed from the data, model coefficients, or trend line calculations. Outlier data records are removed when the error values are greater than or equal to the user-supplied criteria. For optimization methods or other iterative calculations, the removed data are re-applied each iteration to the model computing new results. Using model values for the complete dataset, new error values are computed and the outlier bias reduction procedure is re-applied. Overall error is minimized for model coefficients and outlier removed data in an iterative fashion until user defined error improvement limits are reached. The filtered data may be used for validation, outlier bias reduction and data quality operations.

We claim: 
     
         1 . A computer implemented method comprising the steps of:
 electronically receiving, by a specially programmed computing system, a target data set comprising all actual data values collected for at least one target variable;   generating, by the specially programmed computing system, a random data set based on the target data set;   electronically receiving, by the specially programmed computing system, a plurality of bias criteria values used to determine one or more outliers;   generating, by the specially programmed computing system, a plurality of outlier bias reduced target data sets for each of the bias criteria values using the target data set and a model;   generating, by the specially programmed computing system, a plurality of outlier bias reduced random data sets for each of the bias criteria values using the random data set and the model;   calculating, by the specially programmed computing system, at least one target error value for each of the outlier bias reduced target data sets and at least one random error value for each of the outlier bias reduced random data sets;   calculating, by the specially programmed computing system, at least one target correlation value for each of the outlier bias reduced target data sets and at least one random correlation value for each of the outlier bias reduced random data sets;   constructing, by the specially programmed computing system, a first bias criteria curve for the target data set based on the at least one target error value and the at least one target correlation value for each of the outlier bias reduced target data sets;   constructing, by the specially programmed computing system, a second bias criteria curve for the random data set on based on the at least one random error value and the at least one random correlation value for each of the outlier bias reduced random data sets; and   comparing, by the specially programmed computing system, the first bias criteria curve and the second bias criteria curve to determine viability of the target data set used to develop the model.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the random data set comprises a plurality of random data values generated within a range of a plurality of predicted values of the model. 
     
     
         3 . The computer-implemented method of  claim 1 , further comprising translating, by the specially programmed computing system, the comparison of the first bias criteria curve and the second bias criteria curve to an automated advice message that indicates the viability of the target data set used to develop the model. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the at least one target error value is a standard error, and wherein the at least one target correlation value is a coefficient of determination value. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein generating, by the specially programmed computing system, the plurality of outlier bias reduced target data sets for each of the bias criteria values using the target data set and the model comprises:
 for each of the bias criteria values:
 generating, by the specially programmed computing system, a plurality of model predicted values for the target data set by applying the model to the target data set; 
 computing, by the specially programmed computing system, a plurality of error values using the target data set and the model predicted values; 
 comparing, by the specially programmed computing system, the error values with the corresponding bias criteria value; 
 removing, by the specially programmed computing system, outliers within the target data set to form the corresponding outlier bias reduced target data set determined from the comparison of the error values with the corresponding bias criteria value; and 
 optimizing, by the specially programmed computing system, the model to form an updated model based on the corresponding outlier bias reduced target data set. 
   
     
     
         6 . The computer-implemented method of  claim 5 , wherein the error values comprises a plurality of relative error values and a plurality of absolute error values. 
     
     
         7 . The computer-implemented method of  claim 5 , wherein generating, by the specially programmed computing system, the plurality of outlier bias reduced target data sets for each of the bias criteria values using the target data set further comprises:
 for each of the bias criteria values:
 comparing, by the specially programmed computing system, the error values with a predefined termination criteria to determine termination of optimizing the model; and 
 generating, by the specially programmed computing system, a plurality of second model predicted values for the target data set by applying the updated model to the target data set when the comparison of the error values and the predefined termination criteria do not represent termination of optimizing the model. 
   
     
     
         8 . The computer-implemented method of  claim 1 , wherein comparing, by the specially programmed computing system, the first bias criteria curve and the second bias criteria curve for determining viability of the target data set used to develop the model comprises:
 determining, by the specially programmed computing system, a first bias criteria value on the first bias criteria curve based on the at least one target error value;   determining, by the specially programmed computing system, a second bias criteria value on the second bias criteria curve based on the at least one random error value; and   comparing, by the specially programmed computing system, the first bias criteria value with the second bias criteria value, wherein the at least on target error value and the at least one random error value are the same.   
     
     
         9 . The computer-implemented method of  claim 1 , further comprising:
 determining an influence of the outlier bias reduction for each bias criteria value by comparing a number of iterations to optimize the updated model for each of the bias criteria values; and   comparing differences in the at least one target correlation value for each of the bias criteria values.   
     
     
         10 . The computer-implemented method of  claim 1 , wherein the random data set comprises all random data values based on the target data set, and wherein generating, by the specially programmed computing system, the plurality of outlier bias reduced random data sets for each of the bias criteria values using the random data set and the model comprises:
 for each of the bias criteria values:
 generating, by the specially programmed computing system, a plurality of model predicted values for the random data set by applying the model to the random data set; 
 computing, by the specially programmed computing system, a plurality of error values using the random data set and the model predicted values; 
 comparing, by the specially programmed computing system, the error values with the corresponding bias criteria value; 
 removing, by the specially programmed computing system, outliers within the random data set to form the corresponding outlier bias reduced random data set determined from the comparison of the error values with the corresponding bias criteria value; and 
 optimizing, by the specially programmed computing system, the model for form an updated model based on the corresponding outlier bias reduced random data set. 
   
     
     
         11 . A system specialized for assessing the viability of a data set as used in developing a model comprising:
 a server, comprising a processor and a non-transient storage subsystem;   a database stored by the non-transient storage subsystem comprising a target data set comprising all actual data values collected for at least one target variable;   a computer program stored by the non-transient storage subsystem comprising instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing a model to perform at least the following:
 generate a random data set from the target data set; 
 obtain a set of bias criteria values used to determine one or more outliers; 
 perform dynamic outlier bias reduction on the target data set for one or more bias criteria values of the set of bias criteria values to generate one or more outlier bias reduced target data sets; 
 perform dynamic outlier bias reduction on the random data set for the one or more bias criteria values of the set of bias criteria values to generate one or more outlier bias reduced random data sets; 
 calculate a set of target error values for the one or more outlier bias reduced target data sets and a set of random error values for the one or more outlier bias reduced random data sets; 
 calculate a set of target correlation coefficients for the one or more outlier bias reduced target data sets and a set of random correlation coefficients for the outlier bias reduced random data set; 
 construct a first bias criteria curve for the target data set and a second bias criteria curve for the random data set from the one or more bias criteria values, the set of target error values, the set of random error values, the set of target correlation coefficients, and the set of random correlation coefficients; and 
 compare the first bias criteria curve and the second bias criteria curve for determining viability of the target data set used to develop the model. 
   
     
     
         12 . The system of  claim 11 , wherein the instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing the model to translate the comparison of the first bias criteria curve and the second bias criteria curve to an automated advice message that indicates the viability of the target data set used to develop the model. 
     
     
         13 . The system of  claim 11 , wherein the instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing the model to perform dynamic outlier bias reduction on the target data set for one or more bias criteria values of the set of bias criteria values to generate one or more outlier bias reduced target data sets by performing at least the following:
 for each of the one or more bias criteria values:
 generate a plurality of model predicted values for the target data set by applying the model to the target data set; 
 compute a plurality of error values determined from the target data set and the model predicted values; 
 compare the error values with the corresponding bias criteria value; 
 remove outliers within the target data set to form the corresponding outlier bias reduced target data set determined from the comparison of the error values with the corresponding bias criteria value; and 
 optimize the model to from an updated model determined from the corresponding outlier bias reduced target data set. 
   
     
     
         14 . The system of  claim 13 , wherein the instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing the model to perform dynamic outlier bias reduction on the target data set for one or more bias criteria values of the set of bias criteria values to generate one or more outlier bias reduced target data sets by performing at least the following:
 for each of the one or more bias criteria values:
 compare the error values with a predefined termination criteria to determine termination of optimizing the model; and 
 generate a plurality of second model predicted values for the target data set by applying the updated model to the target data set when the comparison of the error values and the predefined termination criteria do not represent termination of optimizing the model. 
   
     
     
         15 . The system of  claim 11 , wherein the instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing the model to compare the first bias criteria curve and the second bias criteria curve for determining viability of the target data set used to develop the model by performing at least the following:
 determining a first bias criteria value on the first bias criteria curve that corresponds to a first target error value of the set of target error values;   determining a second bias criteria value on the second bias criteria curve that corresponds to a first random error value of the set of random error values; and   comparing the first bias criteria value with the second bias criteria value, wherein the first target error value and the first random error value are the same.   
     
     
         16 . The system of  claim 11 , wherein the instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing the model to determine an influence of the dynamic outlier bias reduction for each bias criteria value by comparing a number of iterations to optimize the model for each of the bias criteria values and comparing the differences in the set of target correlation coefficients. 
     
     
         17 . The system of  claim 11 , wherein the instructions, when executed by the processor, cause the system specialized for assessing the viability of the target data set for developing the model to perform dynamic outlier bias reduction on the target data set for one or more bias criteria values of the set of bias criteria values to generate one or more outlier bias reduced target data sets by performing at least the following:
 for each of the one or more bias criteria values:
 generate a plurality of model predicted values for the target data set by applying the model to the target data set; 
 compute a plurality of error values determined from the target data set and the model predicted values; 
 compare the error values with the corresponding bias criteria value; 
 remove outliers within the target data set to form the corresponding outlier bias reduced target data set determined from the comparison of the error values with the corresponding bias criteria value; and 
 optimize the model to from an updated model determined from the corresponding outlier bias reduced target data set. 
   
     
     
         18 . An apparatus specialized for assessing the viability of a data set for developing a model comprising:
 a processor coupled to a non-transitory computer readable medium, wherein the non-transitory computer readable medium comprises instructions when executed by the processor causes the apparatus to perform at least the following:
 receive a target data set comprising all actual data values collected for at least one target variable; 
 generate a random data set based on the target data set; 
 receive a plurality of bias criteria values used to determine one or more outliers; 
 produce a plurality of outlier bias reduced target data sets that are associated with the bias criteria values by applying a mathematical model and a dynamic outlier bias reduction to the target data set; 
 produce a plurality of outlier bias reduced random data sets that are associated with the bias criteria values by applying the mathematical model and the dynamic outlier bias reduction to the random data set; 
 calculate at least one target error value for each of the outlier bias reduced target data sets and at least one random error value for each of the outlier bias reduced random data sets; 
 calculate at least one target correlation value for each of the outlier bias reduced target data sets and at least one random correlation value for each of the outlier bias reduced random data sets; 
 construct a first bias criteria curve for the target data set on based on the at least one target error value and the at least one target correlation value for each of the outlier bias reduced target data sets; 
 construct a second bias criteria curve for the random data set on based on the at least one random error value and the at least one random correlation value for each of the outlier bias reduced random data sets; and 
 compare the first bias criteria curve and the second bias criteria curve to determine viability of the target data set used for the mathematical model. 
   
     
     
         19 . The apparatus of  claim 18 , wherein the instructions, when executed by the processor, cause the apparatus to produce a plurality of outlier bias reduced target data sets that are associated with the bias criteria values by applying a mathematical model and a dynamic outlier bias reduction to the target data set by performing at least the following:
 for each of the one or more bias criteria values:
 generate a plurality of model predicted values for the target data set by applying the mathematical model to the target data set; 
 compute a plurality of error values determined from the target data set and the model predicted values; 
 compare the error values with the corresponding bias criteria value; 
 remove outliers within the target data set to form the corresponding outlier bias reduced target data set determined from the comparison of the error values with the corresponding bias criteria value; and 
 optimize the mathematical model to from an updated mathematical model determined from the corresponding outlier bias reduced target data set. 
   
     
     
         20 . The apparatus of  claim 18 , wherein the instructions, when executed by the processor, cause the apparatus to compare the first bias criteria curve and the second bias criteria curve for determining viability of the target data set used to develop the model by performing at least the following:
 determine a first bias criteria value on the first bias criteria curve that corresponds to the at least one target error value;   determine a second bias criteria value on the second bias criteria curve that corresponds to the at least one random error values, and   compare the first bias criteria value with the second bias criteria value, wherein the at least one target error value and the at least one random error value are the same.