DPOD: differentially private outsourcing of anomaly detection

Abstract

A method, system and apparatus are disclosed. According to one or more embodiments, a data node is provided. The data node includes processing circuitry configured to: receive an anomaly estimation for a first privatized dataset, the first private dataset being based on a dataset and a first noise profile, apply a second noise profile to the dataset to generate a second privatized dataset, the second noise profile being based at least on the anomaly estimation, and optionally cause transmission of the second privatized dataset for anomaly estimation.

What is claimed is: 
     
       1. A data node configured to communicate with a data analyzer node and a plurality of data owners, the data node comprising processing circuitry configured to:
 receive from each data owner of the plurality of data owners, a dataset having a respective privacy requirement and a sensitivity; 
 apply to each dataset a respective one of a plurality of first noise profiles, each first noise profile configured to add noise to a respective dataset according to the respective privacy requirements of the dataset to provide a plurality of first privatized datasets; 
 release the plurality of first privatized data sets to the data analyzer node; 
 receive from the data analyzer node an anomaly estimation for each first privatized dataset, the anomaly estimation being indicative of outlier data in a respective first privatized dataset; 
 update the respective sensitivity for each first privatized dataset based at least in part on the anomaly estimations; 
 apply a second noise profile configured to reduce noise of outlier data of each first privatized dataset to generate respective second privatized datasets, the second noise profile being selected based at least in part on the respective updated sensitivity. 
 
     
     
       2. The data node of  claim 1 , wherein the processing circuitry is further configured to determine an updated sensitivity value for the dataset based at least on the anomaly estimation and a privacy budget. 
     
     
       3. The data node of  claim 1 , wherein a received dataset includes outlier data and non-outlier data, the second noise profile being configured to reduce noise applied to outlier data when compared to a noise applied to outlier data by the first noise profile. 
     
     
       4. The data node of  claim 1 , wherein a received dataset includes outlier data and non-outlier data, the second noise profile being configured to increase noise applied to non-outlier data when compared to noise applied to the non-outlier data by the first noise profile. 
     
     
       5. The data node of  claim 1 , where the processing circuitry is further configured to apply the first noise profile to outlier data in a received dataset; and
 applying the second noise profile to a first privatized dataset to generate the second privatized dataset includes applying the second noise profile to non-outlier data in the dataset. 
 
     
     
       6. The data node of  claim 1 , wherein the anomaly estimation indicates an anomaly score for a respective privatized dataset. 
     
     
       7. The data node of  claim 1 , wherein the processing circuitry is further configured to receive a sensitivity estimation that is based on the first privatized dataset, the sensitivity estimation indicating whether to modify the first noise profile, the second noise profile being further based at least on the sensitivity estimation. 
     
     
       8. The data node of  claim 7 , wherein the sensitivity estimation indicates to reduce privacy for outlier data of the dataset while at least maintaining privacy for non-outlier data of the dataset. 
     
     
       9. The data node of  claim 1 , wherein a first noise profile provides a first data sensitivity value for a first privatized dataset; and
 the second noise profile provides a second data sensitivity value for the second privatized dataset, the second data sensitivity value being different from the first data sensitivity value. 
 
     
     
       10. The data node of  claim 1 , wherein respective noise profiles of the plurality of first noise profiles each correspond to a respective differential privacy mechanism configured to quantify a privacy level provided to data of the respective received dataset. 
     
     
       11. The data node of  claim 1 , wherein a first privatized dataset is a first privatized histogram of the respective received dataset and the second privatized dataset is a second privatized histogram of the first privatized dataset. 
     
     
       12. A data analyst node configured to communicate with a data node, the data analyst node comprising processing circuitry configured to:
 receive from the data node a plurality of first privatized data sets, each first privatized data set being based on application of a respective first noise profile of a plurality of noise profiles configured to add noise to a respective dataset received by the data node from a different data owner of a plurality of data owners, each received respective dataset having a respective privacy requirement and a sensitivity; 
 determine an anomaly estimation for each first privatized dataset, the anomaly estimation being indicative of outlier data in a respective first privatized dataset; 
 transmit to the data node the anomaly estimation for a first privatized dataset; 
 receive a second privatized dataset, the second privatized dataset being based on application of a second noise profile configured to reduce noise of outlier data of a first privatized dataset based at least in part on an updated sensitivity for the first privatized database, the second noise profile being based at least on the anomaly estimation; and 
 perform anomaly estimation for the second privatized dataset. 
 
     
     
       13. The data analyst node of  claim 12 , wherein a received first privatized dataset includes outlier data and non-outlier data, the second noise profile being configured to reduce noise applied to outlier data when compared to a noise applied to outlier data by the first noise profile. 
     
     
       14. The data analyst node of  claim 12 , wherein a received first privatized dataset includes outlier data and non-outlier data, the second noise profile being configured to increase noise applied to non-outlier data when compared to noise applied to the non-outlier data by the first noise profile. 
     
     
       15. The data analyst node of  claim 12 , wherein the anomaly estimation indicates an anomaly score for a respective privatized dataset. 
     
     
       16. The data analyst node of  claim 12 , wherein a respective first noise profile provides a first data sensitivity value for a respective first privatized dataset; and
 the second noise profile provides a second data sensitivity value for the second privatized dataset, the second data sensitivity value being different from the first data sensitivity value. 
 
     
     
       17. The data analyst node of  claim 12 , wherein the processing circuitry is further configured to determine a sensitivity estimation that is based on a first privatized dataset, the sensitivity estimation indicating whether to modify the respective first noise profile, the second noise profile being further based at least on the sensitivity estimation. 
     
     
       18. The data analyst node of  claim 12 , wherein the sensitivity estimation indicates to reduce privacy for outlier data of the dataset while at least maintaining privacy for non-outlier data of the dataset. 
     
     
       19. The data analyst node of  claim 12 , wherein the processing circuitry is further configured to cause transmission of the anomaly estimation for the second privatized dataset. 
     
     
       20. The data analyst node of  claim 12 , wherein a first privatized dataset is a first privatized histogram of the first privatized dataset and the second privatized dataset is a second privatized histogram of the dataset. 
     
     
       21. A method implemented by a data node configured to communicate with a data analyzer node and a plurality of data owners, the method comprising:
 receiving from each data owner of the plurality of data owners, a dataset having a respective privacy requirement and a sensitivity; 
 applying to each dataset a respective one of a plurality of first noise profiles, each first noise profile configured to add noise to a respective dataset according to the respective privacy requirements of the dataset to provide a plurality of first privatized datasets; 
 releasing the plurality of first privatized data sets to the data analyzer node; 
 receiving from the data analyzer node an anomaly estimation for each first privatized dataset, the anomaly estimation being indicative of outlier data in a respective first privatized dataset; 
 updating the respective sensitivity for each first privatized dataset based at least in part on the anomaly estimations; 
 applying a second noise profile to the configured to reduce noise of outlier data of each first privatized dataset to generate respective second privatized datasets, the second noise profile being selected based at least in part on respective updated sensitivity. 
 
     
     
       22. The method of  claim 21 , further comprising determining an updated sensitivity value for the dataset based at least on the anomaly estimation and privacy. 
     
     
       23. The method of  claim 21 , wherein a received dataset includes outlier data and non-outlier data, the second noise profile being configured to reduce noise applied to outlier data when compared to a noise applied to outlier data by the first noise profile. 
     
     
       24. The method of  claim 21 , wherein a received dataset includes outlier data and non-outlier data, the second noise profile being configured to increase noise applied to non-outlier data when compared to noise applied to the non-outlier data by the first noise profile. 
     
     
       25. The method of  claim 21 , further comprising applying the first noise profile to outlier data in a received dataset; and
 applying of the second noise profile to a first privatized dataset to generate the second privatized dataset includes applying the second noise profile to non-outlier data in the dataset. 
 
     
     
       26. The method of  claim 21 , wherein the anomaly estimation indicates an anomaly score for a respective privatized dataset. 
     
     
       27. The method of  claim 21 , further comprising receiving a sensitivity estimation that is based on a first privatized dataset, the sensitivity estimation indicating whether to modify a first noise profile, the second noise profile being further based at least on the sensitivity estimation. 
     
     
       28. The method of  claim 27 , wherein the sensitivity estimation indicates to reduce privacy for outlier data of a second privatized dataset while at least maintaining privacy for non-outlier data of the dataset. 
     
     
       29. The method of  claim 21 , wherein a first noise profile provides a first data sensitivity value for a first privatized dataset; and
 the second noise profile provides a second data sensitivity value for the second privatized dataset, the second data sensitivity value being different from the first data sensitivity value. 
 
     
     
       30. The method of  claim 21 , wherein the respective noise profiles of the plurality of first noise profiles each correspond to a respective differential privacy mechanism configured to quantify a privacy level provided to data of the respective received dataset. 
     
     
       31. The method of  claim 21 , wherein a first privatized dataset is a first privatized histogram of the respective received dataset and the second privatized dataset is a second privatized histogram of the first privatized dataset. 
     
     
       32. A method implemented by a data analyst node configured to communicate with a data node, the method comprising:
 receiving from the data node a plurality of first privatized data sets, each first privatized data set being based on application of a respective first noise profile of a plurality of noise profiles configured to add noise to a respective dataset received by the data node from a different data owner of a plurality of data owners, each received respective dataset having a respective privacy requirement and a sensitivity; 
 determining an anomaly estimation for each first privatized dataset, the anomaly estimation being indicative of outlier data in a respective first privatized dataset; 
 transmit the anomaly estimation for a first privatized dataset; 
 receiving a second privatized dataset the second privatized dataset being based on application of a second noise profile configured to reduce noise of outlier data of a first privatized dataset based at least in part on an updated sensitivity for the first privatized database, the second noise profile being based at least on the anomaly estimation; and 
 performing anomaly estimation for the second privatized dataset. 
 
     
     
       33. The method of  claim 32 , wherein a received first privatized dataset includes outlier data and non-outlier data, the second noise profile being configured to reduce noise applied to outlier data when compared to a noise applied to outlier data by the first noise profile. 
     
     
       34. The method of  claim 32 , wherein a received first privatized dataset includes outlier data and non-outlier data, the second noise profile being configured to increase noise applied to non-outlier data when compared to noise applied to the non-outlier data by the first noise profile. 
     
     
       35. The method of  claim 32 , wherein the anomaly estimation indicates an anomaly score for a respective privatized dataset. 
     
     
       36. The method of  claim 32 , wherein a respective first noise profile provides a first data sensitivity value for a respective first privatized dataset; and
 the second noise profile provides a second data sensitivity value for the second privatized dataset, the second data sensitivity value being different from the first data sensitivity value. 
 
     
     
       37. The method of  claim 32 , further comprising determining a sensitivity estimation that is based on a first privatized dataset, the sensitivity estimation indicating whether to modify the respective first noise profile, the second noise profile being further based at least on the sensitivity estimation. 
     
     
       38. The method of  claim 32 , wherein the sensitivity estimation indicates to reduce privacy for outlier data of the second privatized dataset while at least maintaining privacy for non-outlier data of the second privatized dataset. 
     
     
       39. The method of  claim 32 , further comprising causing transmission of the anomaly estimation for the second privatized dataset. 
     
     
       40. The method of  claim 32 , wherein a first privatized dataset is a first privatized histogram of the first privatized dataset and the second privatized dataset is a second privatized histogram of the dataset.