Robust modeling

Abstract

A system and method are disclosed for generating a robust model of a system that selects a modeling function. The modeling function has a set of weights and the modeling function has a complexity that is determined by a complexity parameter. For each of a plurality of values of the complexity parameter an associated set of weights of the modeling function is determined such that a training error is minimized for a training data set. An error for a cross validation data set is determined for each set of weights associated with one of the plurality of values of the complexity parameter and the set of weights associated with the value of the complexity parameter is selected that best satisfies a cross validation criteria. Thus, the selected set of weights used with the modeling function provides the robust model.

1. A computer-implemented method of generating a robust model of a system comprising:
 selecting a modeling function having a set of weights, wherein the modeling function has a complexity that is determined by a complexity parameter; 
 receiving, via an input interface, model specification data of the modeling function for each of a plurality of values of the complexity parameter,; 
 retrieving a training data set from a memory; 
 determining an associated set of weights of the modeling function such that a training error is minimized for a the training data set; 
 determining an error for a cross validation data set for each set of weights associated with one of the plurality of values of the complexity parameter; and 
 selecting the set of weights associated with a value of the complexity parameter that best satisfies a cross validation criteria;, whereby the selected set of weights used with the modeling function provides the robust model, 
 wherein the cross validation criteria comprises maximizing lift for the data in the cross validation set; and 
 outputting the set weights via an output interface. 
 
     
     
       2. A method of generating a robust model of a system as recited in  claim 1  wherein the training error is calculated using a training error criteria that is a function of a difference between training output values associated with training input values determined from the training data set and output values determined from the modeling function and the associated set of weights applied to the training input values. 
     
     
       3. A method of generating a robust model of a system as recited in  claim 1  wherein the complexity parameter affects how the training error is minimized. 
     
     
       4. A method of generating a robust model of a system as recited in  claim 3  wherein the complexity parameter causes the training error to be decreased for sets of weights that are more complex. 
     
     
       5. A method of generating a robust model of a system as recited in  claim 4  wherein the complexity of a modeling function having a set of weights is determined by squared weights of said set. 
     
     
       6. A method of generating a robust model of a system as recited in  claim 1  wherein the complexity parameter is a regularization factor. 
     
     
       7. A method of generating a robust model of a system as recited in  claim 1  wherein the complexity parameter controls an amount of noise that is added to input data of the training set. 
     
     
       8. A method of generating a robust model of a system as recited in  claim 1  wherein the modeling function is a first order polynomial. 
     
     
       9. A method of generating a robust model of a system as recited in  claim 1  wherein the modeling function is a second order polynomial. 
     
     
       10. A method of generating a robust model of a system as recited in  claim 1  wherein the modeling function is a second order polynomial that includes cross products between input values. 
     
     
       11. A method of generating a robust model of a system as recited in  claim 1  wherein the plurality of values of the complexity parameter are selected to best satisfy the cross validation criteria using a Newtonian minimization scheme. 
     
     
       12. A method of generating a robust model of a system as recited in  claim 1  wherein the plurality of values of the complexity parameter are selected to best satisfy the cross validation criteria using a Brent method. 
     
     
       13. A method of generating a robust model of a system as recited in  claim 1  further including separating an empirical data set into a training data set and a cross validation data set. 
     
     
       14. A method of generating a robust model of a system as recited in  claim 1  wherein a threshold is applied to an output of the robust model to classify a set of inputs that generated the output of the robust model. 
     
     
       15. A method of generating a robust model of a system as recited in  claim 1  wherein the training error for a training data set having input elements and output elements is defined as a sum of squared differences between said output elements and outputs of the modeling function associated with corresponding ones of said input elements. 
     
     
       16. A method of generating a robust model of a system as recited in  claim 1  wherein the training error for a training data set having input elements and output elements is defined as a sum of differences between said output elements and outputs of the modeling function associated with corresponding ones of said input elements. 
     
     
       17. A method of generating a robust model of a system as recited in  claim 1  wherein the training error for a training data set having input elements and output elements is defined as a maximum difference between output elements of the training data and outputs of the modeling function associated with corresponding ones of said input elements. 
     
     
       18. A method of generating a robust model of a system as recited in  claim 1  further including normalizing the training data. 
     
     
       19. A method of generating a robust model of a system as recited in  claim 1  further including splitting a set of data into a training data set and a cross validation training set. 
     
     
       20. A method of generating a robust model of a system as recited in  claim 1  further including recalculating the set of weights using both the training data set and the cross validation data set. 
     
     
       21. A method of generating a robust model of a system as recited in  claim 1  wherein the cross validation criteria is maximizing lift. 
     
     
       22. A method of generating a robust model of a system as recited in  claim 1  wherein the cross validation criteria is minimizing a measure of error between the robust model and the cross validation set. 
     
     
       23. A method of generating a robust model of a system comprising:
 selecting a modeling function having a set of weights wherein the modeling function has a complexity that is determined by a complexity parameter;   for a each of a plurality of values of the complexity parameter, determining an associated set of weights of the modeling function such that a training error is minimized for a training data set;   determining a cross validation error for a cross validation data set for each set of weights associated with one of the plurality of values of the complexity parameter;   determining an optimal value of the complexity parameter that minimizes the cross validation error; and   determining an output set of weights of the modeling function using the optimal value of the complexity parameter and an aggregate training data set that includes a training data set and the cross validation data set such that an aggregate training error is minimized for the aggregate training data set; and   whereby the output set of weights used with the modeling function provides the robust model.   
     
     
       24. A robust modeling engine comprising:
 a memory configured to store a training data set and a cross validation data set; 
 an input interface configured to receive model specification data of a modeling function; 
 a processor configured to:
 select a modeling function having a set of weights, wherein the modeling function has a complexity that is determined by a complexity parameter; 
 for each of a plurality of values of the complexity parameter, determine an associated set of weights of the modeling function such that a training error is minimized for a training data set; 
 determine an error a quantity for a cross validation data set for each set of weights associated with one of the plurality of values of the complexity parameter; and 
 select the set of weights associated with the complexity parameter that best satisfies a cross validation criteria; and 
 
 an output interface configured to output the set of weights associated with the value of the complexity parameter that best satisfies a cross validation criteria, wherein the cross validation criteria comprises maximizing lift.