Computer Implemented Tool and Method for Automating the Forecasting Process

Abstract

A tool and method for automating the process of forecasting are disclosed. The tool includes a data collector for collecting data from a plurality of sources. An extractor extracts, from the collected data, the predictor data streams and elements to be used for forecasting. The predictor data streams and elements are analyzed for completeness and their corresponding characteristics are determined. Further, the predictor data streams and elements are arranged into multiple sets such that each set has a common data pattern. A plurality of forecasting models and the corresponding unique properties and a unique master list of their forecasting parameters, are stored in a second repository. The unique properties are compared with the properties of each of the forecasting models, and a set of forecasting models suitable for performing a forecast are identified and ranked. The most appropriate forecasting model is selected and provided to the forecasting engine, for forecasting.

1 . A computer implemented tool for automating the forecasting process, said tool comprising:
 a data collector configured to collect data from a plurality of sources including historic time series data, causal information and trend indicators, said data collector further configured to collect data corresponding to external factors including macroeconomic factors, market related factors and social media related factors;   an extractor configured to extract, from said collected data, the predictor data streams and elements to be utilized for forecasting;   a first repository configured to store the extracted predictor data streams and elements;   a first analyzer adapted to receive said predictor data streams and elements from said repository, said first analyzer configured to analyze said predictor data streams and elements, and determine the characteristics of said predictor data streams and elements, said analyzer further configured to determine iteratively, in accordance with pre-determined rules, whether all predictor data streams and elements required for forecasting are present, and create sets of predictor data streams and elements which are appropriate for forecasting;   a clustering module provided with a set of clustering and pattern recognition rules, said clustering module configured to receive sets of appropriate predictor data streams and elements analyzed by said analyzer, and group said predictor data streams and elements into a plurality of clusters in accordance with said set of clustering rules such that each cluster contains appropriate predictor data streams and elements having a common data pattern, said clustering module further adapted to extract the characteristic(s) corresponding to each cluster;   a second repository storing a plurality of forecasting models and the corresponding unique properties and a unique master list of their forecasting parameters, for each of the plurality of forecasting models;   a classifier provided with a set of classifying rules, said classifier configured to receive said properties of each of said forecasting models and the extracted characteristics of predictor data streams and elements present in each of said clusters, said classifier further having a comparator configured to compare said characteristics with said properties and identify a set of forecasting models containing at least one appropriate forecasting model adapted to perform a forecast using said predictor data streams and elements;   a ranking module provided with a set of ranking rules and configured to receive said set of forecasting models, said ranking module having a ranking processor adapted to process the received set of models in accordance with said set of ranking rules and thereby rank the set of forecasting models, in accordance with their ability to match with the characteristics of the predictor data elements and data streams;   a selection module configured to receive said set of ranked forecasting models and select a ranked forecasting model;   a forecasting engine configured to receive the selected ranked forecasting model and the master list of its forecasting parameters, said forecasting engine further comprising:
 a value determinator adapted to receive said forecasting parameters of the selected forecasting model; 
 a second analyzer cooperating with said value determinator and configured to apply shrinkage based techniques to eliminate inter-dependent predictor variables, in the event that the selected forecasting model is a regression model, said second analyzer further configured to apply state-space modeling to automate the determination of optimal parameter values, in the event that the selected forecasting model is an exponential smoothing model, said second analyzer configured to automatically select an appropriate set of forecasting parameters and corresponding optimal parameter values; and 
 a processor configured to process said appropriate set of forecasting parameters and the corresponding optimal parameter values in accordance with the selected forecasting model, to generate a forecast. 
   
     
     
         2 . The tool as claimed in  claim 1 , wherein said tool further includes a self-correcting error diagnosing module, said error diagnosing module comprising:
 a first identifier configured to identify at least one forecast error criteria for the selected forecasting model and calculate the forecast error based on a subset of the forecast error criteria;   a third analyzer configured to analyze the forecast bias and forecast error pattern corresponding to the forecast error criteria and determine their characteristics;   a linker configured to identify the properties of the forecast error criteria including the forecast bias and forecast error pattern, said linker having access to the forecasting parameters corresponding to the selected forecasting model, said linker further configured to link said forecast error criteria with said forecasting parameters based on the properties of said forecast error criteria, said linker further configured to identify a set of potential error causing parameters;   a simulator adapted to receive said set of potential error causing parameters and configured to vary the values of each of said error causing parameters within a pre-determined range to achieve forecast error attenuation, said simulator further configured to identify the sequence in which the values of each of the error causing parameters should be modified, based on the sensitivity of said error causing parameters and the impact on the stability of the selected forecast model;   a third repository adapted to store a plurality of forecast error scenarios wherein each error scenario is defined by its corresponding unique error characteristics, the third repository further adapted to receive from the simulator and store the error causing parameters modified by said simulator for a selected forecasting model, the third repository further adapted to store a map between the modified error causing parameters and the respective forecast error scenarios, the computer implemented tool including a mapper adapted to perform the mapping;   an error correction engine configured to receive said forecast error scenarios from said third repository, the error correction engine further configured to evaluate the received forecast error scenarios and match each of the received forecast error scenarios with the modified error causing parameters stored in the third repository and identify the modified error causing parameters that bring about the occurrence of each of the forecasting scenarios, the error correction engine further configured to define a heuristic that relates the unique characteristics of each of said forecast error scenarios with the modified error causing parameters of the selected forecasting model, the error correction engine further configured to suggest optimal values for the parameters which provide for the most consistent error reduction, for a received forecast error scenario.   
     
     
         3 . The tool as claimed in  claim 1 , wherein the characteristics corresponding to the predictor data streams and elements are selected from the group of characteristics consisting of trend related characteristics, season related characteristics and cycle related characteristics. 
     
     
         4 . The tool as claimed in  claim 1 , wherein said clustering and pattern recognition rules are selected from the group consisting of at least k-means clustering technique, Kohonen SOM based clustering technique, and hierarchical techniques. 
     
     
         5 . The tool as claimed in  claim 1 , wherein said first analyzer is configured to iteratively analyze said data streams and elements using a rule selected form the group of rules consisting of co-efficient of variation analysis rules, curvi-linear regression rules, rules for combination of first order derivatives and higher order of derivatives and auto-correlation function rules. 
     
     
         6 . The tool as claimed in  claim 1 , wherein said ranking module is further configured to assign a weighted rank to each of the forecasting models present within said set of forecasting models. 
     
     
         7 . A computer implemented method for automating the process of forecasting, said method comprising the following steps:
 collecting data from a plurality of sources including historic time series data, causal information and trend indicators and collecting data corresponding to external factors including macroeconomic factors, market related factors and social media related factors;   extracting, from the collected data, the predictor data streams and elements to be used for forecasting;   storing the extracted predictor data streams and elements in a first repository;   receiving said predictor data streams and elements from said repository and analyzing the received predictor data streams and elements to identify the characteristics of said predictor data streams and elements;   iteratively determining whether all the predictor data streams and elements required for forecasting are present;   creating sets of predictor data streams and elements appropriate for forecasting from said predictor data streams and elements by selecting, combining and transforming one or more predictor data streams and elements;   receiving sets of predictor data streams and elements and grouping the predictor data streams and elements into a plurality of clusters, in accordance with clustering rules such that each cluster contains predictor data streams and elements having a common data pattern;   extracting said characteristic(s) corresponding to each cluster;   storing a plurality of forecasting models and the corresponding unique properties and a unique master list of their forecasting parameters, for each of the plurality of forecasting models;   receiving the properties of each of said forecasting models and the extracted characteristics of predictor data streams and elements present in each of said clusters;   comparing said characteristics with said properties and identifying a set of forecasting models containing at least one appropriate forecasting model adapted to perform a forecast using said predictor data streams and elements;   processing the set of forecasting models in accordance with a set of ranking rules and ranking the set of forecasting models, in accordance with its ability to match with the characteristics of the predictor data elements and data streams;   selecting a ranked forecasting model; and   receiving, at a forecasting engine, the selected forecasting model and the master list of its forecasting parameters;   applying shrinkage based techniques to eliminate inter-dependent predictor variables, in the event that the selected forecasting model is a regression model, and applying state-space modeling to automate the determination of optimal parameter values, in the event that the selected forecasting model is an exponential smoothing model;   automatically select an appropriate set of forecasting parameters and corresponding optimal parameter values;   processing said appropriate set of forecasting parameters and the corresponding optimal parameter values in accordance with the selected forecasting model, to generate a forecast; and   diagnosing the generated forecast and identifying at least one forecast error criteria corresponding to the generated forecast.   
     
     
         8 . The method as claimed in  claim 7 , wherein the step of diagnosing the generated forecast and identifying at least one forecast error criteria corresponding to the generated forecast, further includes the following steps:
 identifying at least one forecast error criteria for the selected forecasting model and calculating the forecast error based on a subset of the forecast error criteria;   analyzing the forecast bias and forecast error pattern corresponding to the forecast error criteria and determining their characteristics;   linking the forecast bias and the forecast error pattern of the forecast error criteria with said forecasting parameters of the selected forecasting model, based on the properties of the forecast error criteria, and identifying a set of potential error causing parameters;   receiving at a simulator, said set of potential error causing parameters and varying the values of each of said error causing parameters within a pre-determined range leading to attenuation of forecast error;   storing, in a third repository, a plurality of forecast error scenarios wherein each of said error scenarios are defined by unique error characteristics;   storing the modified error causing forecasting parameters of each of the forecasting models, that are likely causes of said forecast error scenarios, and storing a mapping between said modified error causing forecasting parameters and the forecast error scenarios;   receiving at an error correction engine, the forecast error scenarios and evaluating the received forecast error scenarios and matching the received forecast error scenarios with the modified error causing parameters stored in the third repository; and   identifying the modified error causing parameters that give rise to each of the forecast error scenarios and defining a heuristic that relates the corresponding forecast error scenario characteristics to the modified error causing parameters and automatically suggesting the optimal values corresponding to the parameters, which provide for the most consistent error reduction for a received forecast error scenario.   
     
     
         9 . The method as claimed in  claim 7 , wherein the step of ranking further includes the step of assigning weights to the individual properties of each of the forecasting models, and calculating a weighted score for each of the forecasting models. 
     
     
         10 . The method as claimed in  claim 8 , wherein the step of varying the values of each of said error causing parameters further includes the following steps:
 defining the rules for selecting values for respective error causing parameters that provide for consistent error reduction; and   identifying, based on the sensitivity of the forecasting parameters and the impact on the stability of the selected forecast model, the sequence in which said values are to be modified.