System and method for reduction of data transmission by inference optimization and data reconstruction

Abstract

Methods and systems for managing data collection are disclosed. To manage data collection, a system may include a data aggregator and a data collector. The data aggregator may utilize complex inference models to predict the future operation of the data collector, while the data collector may host simpler inference models. The data collector may access inferences from the complex models by obtaining a difference between complex and simple inferences from the data aggregator and locally reconstructing the complex differences. To reduce data transmission, the data collector may transmit a data difference (e.g., a reduced-size representation of a measurement) to the data aggregator using the reconstructed complex inferences. The data aggregator may reconstruct data from the data collectors using the data difference from the data collector and inferences from the complex inference model.

What is claimed is: 
     
         1 . A method for managing data collection in a distributed environment where data is collected in a data aggregator of the distributed environment and from at least a data collector operably connected to the data aggregator via a communication system, comprising:
 obtaining, by the data aggregator, an inference difference, the inference difference being based on:
 a first inference generated by an aggregator inference model, the first inference being intended to match data based on measurements performed by the data collector, and 
 a second inference generated by a twin inference model, the second inference being intended to match data based on measurements performed by the data collector; 
   obtaining, from the data collector, a data difference, the data difference being based on:
 data obtained via a measurement performed by the data collector, and 
 a reconstructed inference, the reconstructed inference being generated by the data collector and matching the first inference generated by the aggregator inference model; 
   reconstructing, by the data aggregator, the data using the data difference and the first inference generated by the data aggregator, the first inference being intended to match the data;   performing an action set based at least in part on the reconstructed data, the action set comprising one or more actions to be performed based on the data obtained by the measurement performed by the data collector, and while the data aggregator does not have access to the data obtained by the data collector.   
     
     
         2 . The method of  claim 1 , further comprising:
 providing, to the data collector, a copy of the inference difference prior to obtaining the data difference,   wherein the data difference is obtained by the data aggregator prior to the data collector being provided with the first inference and the second inference.   
     
     
         3 . The method of  claim 1 , wherein the twin inference model hosted by the data aggregator is a copy of a second twin inference model hosted by the data collector, and the reconstructed inference being based on the inference difference generated by the data aggregator, wherein the inference difference is usable to obtain the first inference based on the second inference to which the data collector has access via the copy of the second twin inference model. 
     
     
         4 . The method of  claim 1 , further comprising:
 making a determination that the data difference falls below a threshold; and   based on that determination:
 treating an aggregator inference model as being accurate, the aggregator inference model being implemented by the data aggregator, and the aggregator inference model being used to obtain the first inference. 
   
     
     
         5 . The method of  claim 1 , further comprising:
 making a determination that the data difference falls outside of a threshold; and   based on that determination:
 treating an aggregator inference model as being inaccurate, the aggregator inference model being implemented by the data aggregator, and the aggregator inference model being used to obtain the first inference. 
   
     
     
         6 . The method of  claim 5 , further comprising:
 when the aggregator inference model is determined as being inaccurate:
 updating the aggregator inference model using training data, the training data comprising a portion of data obtained via a series of measurements performed by the data collector. 
   
     
     
         7 . The method of  claim 1 , further comprising:
 prior to obtaining the inference difference:
 obtaining, by the data aggregator, an aggregator inference model using training data obtained, at least in part, from the data collector. 
   
     
     
         8 . The method of  claim 7 , further comprising:
 prior to obtaining the inference difference:
 obtaining, by the data aggregator, a twin inference model using the training data; and 
 distributing, by the data aggregator, a copy of the twin inference model to the data collector. 
   
     
     
         9 . The method of  claim 8 , wherein the aggregator inference model is not provided to the data collector prior to the data difference being obtained by the data aggregator. 
     
     
         10 . The method of  claim 8 , wherein the twin inference model consumes fewer computing resources than the aggregator inference model during operation. 
     
     
         11 . The method of  claim 8 , wherein a value of the data difference decreases as accuracy of the aggregator inference model increases, and the value of the data difference increases as the accuracy of the aggregator inference model decreases. 
     
     
         12 . The method of  claim 11 , wherein a quantity of bits necessary to communicate the data difference via the communication system decreases as the accuracy of the aggregator inference model increases. 
     
     
         13 . The method of  claim 9 , wherein the action set is not based on any data from measurements performed by the data collector that is transmitted via the communication system to the data aggregator. 
     
     
         14 . The method of  claim 1 , wherein the measurement is performed using a sensor that measures a characteristic of an ambient environment proximate to the data collector, the ambient environment proximate to the data collector being different from an ambient environment proximate to the data aggregator. 
     
     
         15 . The method of  claim 11 , wherein the one or more actions are triggered to be performed based on an ambient environment proximate to the data collector and are independent from the ambient environment proximate to the data aggregator. 
     
     
         16 . A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations for managing data collection in a distributed environment where data is collected in a data aggregator of the distributed environment and from at least a data collector operably connected to the data aggregator via a communication system, the operations comprising:
 obtaining, by the data aggregator, an inference difference, the inference difference being based on:
 a first inference generated by an aggregator inference model, the first inference being intended to match data based on measurements performed by the data collector, and 
 a second inference generated by a twin inference model, the second inference being intended to match data based on measurements performed by the data collector; 
   obtaining, from the data collector, a data difference, the data difference being based on:
 data obtained via a measurement performed by the data collector, and 
 a reconstructed inference, the reconstructed inference being generated by the data collector and matching the first inference generated by the aggregator inference model; 
   reconstructing, by the data aggregator, the data using the data difference and the first inference generated by the data aggregator, the first inference being intended to match the data;   performing an action set based at least in part on the reconstructed data, the action set comprising one or more actions to be performed based on the data obtained by the measurement performed by the data collector, and while the data aggregator does not have access to the data obtained by the data collector.   
     
     
         17 . The non-transitory machine-readable medium of  claim 16 , wherein the operations further comprise:
 providing, to the data collector, a copy of the inference difference prior to obtaining the data difference,   wherein the data difference is obtained by the data aggregator prior to the data collector being provided with the first inference and the second inference.   
     
     
         18 . The non-transitory machine-readable medium of  claim 16 , wherein the twin inference model hosted by the data aggregator is a copy of a second twin inference model hosted by the data collector, and the reconstructed inference being based on the inference difference generated by the data aggregator, wherein the inference difference is usable to obtain the first inference based on the second inference to which the data collector has access via the copy of the second twin inference model. 
     
     
         19 . A data aggregator, comprising:
 a processor; and   a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations for managing data collection in a distributed environment where data is collected in the data aggregator of the distributed environment and from at least a data collector operably connected to the data aggregator via a communication system, the operations comprising:
 obtaining, by the data aggregator, an inference difference, the inference difference being based on: 
 a first inference generated by an aggregator inference model, the first inference being intended to match data based on measurements performed by the data collector, and 
 a second inference generated by a twin inference model, the second inference being intended to match data based on measurements performed by the data collector; 
 obtaining, from the data collector, a data difference, the data difference being based on: 
 data obtained via a measurement performed by the data collector, and 
 a reconstructed inference, the reconstructed inference being generated by the data collector and matching the first inference generated by the aggregator inference model; 
 reconstructing, by the data aggregator, the data using the data difference and the first inference generated by the data aggregator, the first inference being intended to match the data; 
 performing an action set based at least in part on the reconstructed data, the action set comprising one or more actions to be performed based on the data obtained by the measurement performed by the data collector, and while the data aggregator does not have access to the data obtained by the data collector. 
   
     
     
         20 . The data aggregator of  claim 19 , wherein the operations further comprise:
 providing, to the data collector, a copy of the inference difference prior to obtaining the data difference,   wherein the data difference is obtained by the data aggregator prior to the data collector being provided with the first inference and the second inference.