System and method of designing models in a feedback loop

Abstract

A method and system for designing models is disclosed. The method includes selecting a plurality of models for modeling a common event of interest. The method further includes aggregating the results of the models and analyzing each model compared to the aggregate result to obtain comparative information. The method also includes providing the information back to the plurality of models to design more accurate models through a feedback loop.

We claim: 
     
         1 . A method of designing models comprising:
 a. selecting a plurality of models for modeling a common event of interest;   b. aggregating the results of the models;   c. analyzing each model compared to the aggregate result to obtain comparative information; and   d. providing the information back to the plurality of models to design more accurate models through a feedback loop.   
     
     
         2 . The method of  claim 1  wherein the aggregating the results of the models comprises inputting the results of the models and measurement data to create an aggregate result using Bayesian model aggregation (BMA). 
     
     
         3 . The method of  claim 1  wherein the event of interest includes one of the following: CO 2  plumes, power system dynamics, protein structures, weather systems, image recognition tasks, and nuclear safety. 
     
     
         4 . The method of  claim 1  wherein the models are conceptual models. 
     
     
         5 . The method of  claim 1  wherein the models are statistical models. 
     
     
         6 . The method of  claim 2  wherein the analyzing each model compared to the aggregate result comprises at least one of the following: 1) comparing estimate differences between the BMA and each model and 2) analyzing how the BMA utilizes each model in the aggregate based on information criteria. 
     
     
         7 . The method of  claim 6  wherein each of the comparing the differences between the BMA and each model and the analyzing how the BMA utilizes each model further comprises using a statistical method to select certain models. 
     
     
         8 . The method of  claim 7  wherein the statistical method is an analysis of variance statistical method. 
     
     
         9 . The method of  claim 6  wherein models which are based on at least one of analysis of variance and the information criteria are identified and held constant. 
     
     
         10 . The method of  claim 7  wherein information regarding the differences in estimates and the information criteria of each model are fed back to the plurality of models. 
     
     
         11 . A system for designing models comprising:
 a. a plurality of models predicting or estimating an event of interest;   b. a workstation for aggregating the results;   c. means for analyzing each model compared to the aggregate result to obtain comparative information; and   d. a feedback loop to provide the information back to the plurality of models to design more accurate models.   
     
     
         12 . The system of  claim 11  wherein the workstation for aggregating utilizes Bayesian model aggregation (BMA) software that takes as inputs the results of the models and measurement data to create an aggregate result. 
     
     
         13 . The system of  claim 11  wherein the event of interest includes, but is not limited to, the following: CO 2  plumes, power system dynamics, protein structures, weather systems, image recognition tasks, and nuclear safety. 
     
     
         14 . The system of  claim 11  wherein the models are conceptual models. 
     
     
         15 . The system of  claim 11  wherein the models are statistical models. 
     
     
         16 . The system of  claim 11  wherein the means for analyzing each model compared to the aggregate result comprises at least one of the following: 1) comparison of estimate differences between the BMA and each model and 2) analysis of how the BMA utilizes each model in the aggregate based on information criteria. 
     
     
         17 . The system of  claim 16  wherein each of the comparison of differences between the BMA and each model and analysis of how the BMA utilizes each model further comprises use of a statistical method to select certain models. 
     
     
         18 . The system of  claim 17  wherein the statistical method is an analysis of variance statistical method. 
     
     
         19 . The system of  claim 18  wherein models which are based on at least one of analysis of variance and information criteria are identified and held constant. 
     
     
         20 . A method of designing models comprising:
 a. selecting a plurality of models for modeling a net interchange schedule (NIS) for a power system;   b. aggregating the results of the NIS models into an aggregate forecast;   c. analyzing each NIS model compared to the aggregate result to obtain comparative information; and   d. providing the information back to the plurality of NIS models to design more accurate models through a feedback loop.