Universal time-series forecasting with adaptive inputs/outputs for real-world random missing data

Abstract

According to one embodiment, first input features are extracted from received past time-dependent inputs. The first input features are represented at least in part by a first plurality of input time series. A first encoder output array is generated by a first encoder with a first cross-attention mechanism based at least in part on the first input features. The first encoder output array is provided as query, key and value inputs to a pretrained core model with a self-attention mechanism to generate a core model output array. Forecasting results in a forecasting time period are generated by a decoder based at least in part on the core model output array. The forecasting results are represented by one or more output time series.

1 . A method comprising:
 extracting first input features from received past time-dependent inputs, wherein the first input features are represented at least in part by a first plurality of input time series;   generating, by a first encoder with a first cross-attention mechanism based at least in part on the first input features, a first encoder output array;   providing the first encoder output array as query, key and value inputs to a pretrained core model with a self-attention mechanism to generate a core model output array;   generating, by a decoder based at least in part on the core model output array, forecasting results in a forecasting time period, wherein the forecasting results are represented by one or more output time series.   
     
     
         2 . The method of  claim 1 , further comprising:
 extracting second input features from received future time-dependent inputs, wherein the second input features are represented at least in part by a second plurality of input time series;   generating, by a second encoder with a second cross-attention mechanism based at least in part on the second input features, a prompt to the decoder;   wherein the forecasting results are generated by the decoder based further on the prompt generated by the second encoder.   
     
     
         3 . The method of  claim 1 , further comprising:
 generating, by a pre-processing mechanism from received time-independent inputs, a latent array of a predefined shape, wherein the pre-processing mechanism includes a sequence of a first feed forward network for element multiplication, a variable selection mechanism, and a second feed forward network for element multiplication;   providing the latent array of the predefined shape as a query input to one of: a second encoder designated to process future inputs or the first encoder designated to process past inputs.   
     
     
         4 . The method of  claim 1 , wherein the past time-dependent inputs are preprocessed into a key input to the first encoder by a variable selection mechanism and followed by a feed forward network for element multiplication; wherein the past time-dependent inputs are preprocessed into a value input to the first encoder by a feed forward network for matrix multiplication. 
     
     
         5 . The method of  claim 1 , wherein at least one of the first encoder or the pretrained core model includes multiple attention heads. 
     
     
         6 . The method of  claim 1 , wherein the first input features are encoded with relative temporal positional information. 
     
     
         7 . The method of  claim 1 , wherein the first plurality of input time series includes a specific input time series comprising physical sensory data in a specific contiguous time duration; wherein the specific contiguous time duration includes one or more time gaps for which there is no physical sensor data available in the specific time series. 
     
     
         8 . The method of  claim 1 , wherein the forecasting results include predictions of one or more of: future State of Charge (SoC) values of an electric vehicle (EV), future home availabilities of the EV, future electricity demands of a home for the EV, or future electricity generation of the home; wherein the forecasting results are used by an optimization system to generate future electricity charging scheduling events for the EV. 
     
     
         9 . One or more non-transitory computer readable media storing a program of instructions that is executable by one or more computing processors to perform:
 extracting first input features from received past time-dependent inputs, wherein the first input features are represented at least in part by a first plurality of input time series;   generating, by a first encoder with a first cross-attention mechanism based at least in part on the first input features, a first encoder output array;   providing the first encoder output array as query, key and value inputs to a pretrained core model with a self-attention mechanism to generate a core model output array;   generating, by a decoder based at least in part on the core model output array, forecasting results in a forecasting time period, wherein the forecasting results are represented by one or more output time series.   
     
     
         10 . The media of  claim 9 , wherein the program of instructions is executable by the one or more computing processors to perform:
 extracting second input features from received future time-dependent inputs, wherein the second input features are represented at least in part by a second plurality of input time series;   generating, by a second encoder with a second cross-attention mechanism based at least in part on the second input features, a prompt to the decoder;   wherein the forecasting results are generated by the decoder based further on the prompt generated by the second encoder.   
     
     
         11 . The media of  claim 9 , wherein the program of instructions is executable by the one or more computing processors to perform:
 generating, by a pre-processing mechanism from received time-independent inputs, a latent array of a predefined shape, wherein the pre-processing mechanism includes a sequence of a first feed forward network for element multiplication, a variable selection mechanism, and a second feed forward network for element multiplication;   providing the latent array of the predefined shape as a query input to one of: a second encoder designated to process future inputs or the first encoder designated to process past inputs.   
     
     
         12 . The media of  claim 9 , wherein the past time-dependent inputs are preprocessed into a key input to the first encoder by a variable selection mechanism and followed by a feed forward network for element multiplication; wherein the past time-dependent inputs are preprocessed into a value input to the first encoder by a feed forward network for matrix multiplication. 
     
     
         13 . The media of  claim 9 , wherein at least one of the first encoder or the pretrained core model includes multiple attention heads. 
     
     
         14 . The media of  claim 9 , wherein the first input features are encoded with relative temporal positional information. 
     
     
         15 . The media of  claim 9 , wherein the first plurality of input time series includes a specific input time series comprising physical sensory data in a specific contiguous time duration; wherein the specific contiguous time duration includes one or more time gaps for which there is no physical sensor data available in the specific time series. 
     
     
         16 . The media of  claim 9 , wherein the forecasting results include predictions of one or more of: future State of Charge (SoC) values of an electric vehicle (EV), future home availabilities of the EV, future electricity demands of a home for the EV, or future electricity generation of the home; wherein the forecasting results are used by an optimization system to generate future electricity charging scheduling events for the EV. 
     
     
         17 . A system comprising: one or more computing processors; one or more non-transitory computer readable media storing a program of instructions that is executable by the one or more computing processors to perform:
 extracting first input features from received past time-dependent inputs, wherein the first input features are represented at least in part by a first plurality of input time series;   generating, by a first encoder with a first cross-attention mechanism based at least in part on the first input features, a first encoder output array;   providing the first encoder output array as query, key and value inputs to a pretrained core model with a self-attention mechanism to generate a core model output array;   generating, by a decoder based at least in part on the core model output array, forecasting results in a forecasting time period, wherein the forecasting results are represented by one or more output time series.   
     
     
         18 . The system of  claim 17 , wherein the program of instructions is executable by the one or more computing processors to perform:
 extracting second input features from received future time-dependent inputs, wherein the second input features are represented at least in part by a second plurality of input time series;   generating, by a second encoder with a second cross-attention mechanism based at least in part on the second input features, a prompt to the decoder;   wherein the forecasting results are generated by the decoder based further on the prompt generated by the second encoder.   
     
     
         19 . The system of  claim 17 , wherein the program of instructions is executable by the one or more computing processors to perform:
 generating, by a pre-processing mechanism from received time-independent inputs, a latent array of a predefined shape, wherein the pre-processing mechanism includes a sequence of a first feed forward network for element multiplication, a variable selection mechanism, and a second feed forward network for element multiplication;   providing the latent array of the predefined shape as a query input to one of: a second encoder designated to process future inputs or the first encoder designated to process past inputs.   
     
     
         20 . The system of  claim 17 , wherein the past time-dependent inputs are preprocessed into a key input to the first encoder by a variable selection mechanism and followed by a feed forward network for element multiplication; wherein the past time-dependent inputs are preprocessed into a value input to the first encoder by a feed forward network for matrix multiplication. 
     
     
         21 . The system of  claim 17 , wherein at least one of the first encoder or the pretrained core model includes multiple attention heads. 
     
     
         22 . The system of  claim 17 , wherein the first input features are encoded with relative temporal positional information. 
     
     
         23 . The system of  claim 17 , wherein the first plurality of input time series includes a specific input time series comprising physical sensory data in a specific contiguous time duration; wherein the specific contiguous time duration includes one or more time gaps for which there is no physical sensor data available in the specific time series. 
     
     
         24 . The system of  claim 17 , wherein the forecasting results include predictions of one or more of: future State of Charge (SoC) values of an electric vehicle (EV), future home availabilities of the EV, future electricity demands of a home for the EV, or future electricity generation of the home; wherein the forecasting results are used by an optimization system to generate future electricity charging scheduling events for the EV.