Metadata classification

Abstract

Systems and method are disclosed that retrieve data from a data set organized in a plurality of columns. For each column in the plurality of columns, the systems and method generate one or more candidate semantic categories for the column, where each of the one or more candidate semantic categories has a corresponding probability. The systems and method create a feature vector for the column from the one or more candidate semantic categories and the corresponding probabilities. The systems and method determine a semantic category type of the column based on the feature vector. The systems and method anonymize the data in the column based on the semantic category type, which includes replacing more specific data in the column with less specific data based on a data hierarchy that relates the more specific data to the less specific data.

What is claimed is: 
     
       1. A method comprising:
 retrieving data from a data set, wherein the data is organized in a plurality of columns; 
 for each column in the plurality of columns:
 generating one or more candidate semantic categories for the column, wherein each of the one or more candidate semantic categories is included in a plurality of semantic categories; 
 generating a probability for each of the one or more candidate semantic categories by applying a bloom filter to the data of the column; 
 creating a feature vector for the column from the one or more candidate semantic categories and the corresponding probabilities; 
 determining a semantic category of the column based on the feature vector; 
 determining a privacy category based on a designation of the semantic category; and 
 anonymizing the data in the column based on the privacy category to produce anonymized data, wherein the anonymizing comprises replacing more specific data in the column with less specific data based on a data hierarchy of the plurality of semantic categories that relates the more specific data to the less specific data, and wherein the less specific data reduces a likelihood that one or more data subjects are identifiable from the anonymized data; and 
 
 generating an anonymized view of the data set using the anonymized data. 
 
     
     
       2. The method of  claim 1 , wherein the data hierarchy comprises a lower level and a higher level, the method further comprising:
 matching the data from the column with a lower level location in the lower level of the data hierarchy; and 
 identifying the less specific data in the higher level at a higher level location that relates to the lower level location. 
 
     
     
       3. The method of  claim 1 , further comprising:
 determining that the privacy category is an identifier semantic type or a quasi-identifier type; and 
 performing the anonymizing of the data in response to determining that the privacy category is the identifier semantic type or the quasi-identifier type. 
 
     
     
       4. The method of  claim 1 , further comprising:
 inputting the feature vector into a trained machine learning model, wherein the trained machine learning model comprises an encoded threshold and identifies one of the one or more candidate semantic categories having a corresponding probability above the encoded threshold; and 
 assigning the candidate semantic category identified by the trained machine learning model to the column. 
 
     
     
       5. The method of  claim 1 , further comprising:
 anonymizing the data, for each of the plurality of columns on an individual basis, based on the privacy category of the corresponding column. 
 
     
     
       6. The method of  claim 1 , further comprising:
 selecting a column from the plurality of columns; 
 applying the bloom filter with a potential semantic category to the data of that the column selected; and 
 computing the probability for the column selected based on a set of results from the applying of the bloom filter to the data in the column selected. 
 
     
     
       7. The method of  claim 6 , wherein the bloom filter is selected from the group consisting of a whitelist bloom filter and a blacklist bloom filter. 
     
     
       8. A system comprising:
 a set of storage resources; 
 a query processor to:
 retrieve data from a data set, wherein the data is organized in a plurality of columns; 
 for each column in the plurality of columns:
 generate, with the query processor, one or more candidate semantic categories for the column, wherein each of the one or more candidate semantic categories is included in a plurality of semantic categories; 
 generate a probability for each of the one or more candidate semantic categories by applying a bloom filter to the data of the column; 
 create a feature vector for the column from the one or more candidate semantic categories and the corresponding probabilities; 
 determine a semantic category of the column based on the feature vector; 
 determine a privacy category based on a designation of the semantic category; and 
 anonymize the data in the column based on the privacy category to produce anonymized data, wherein the anonymizing comprises replacing more specific data in the column with less specific data based on a data hierarchy of the plurality of semantic categories that relates the more specific data to the less specific data, and wherein the less specific data reduces a likelihood that one or more data subjects are identifiable from the anonymized data; and 
 
 generate an anonymized view of the data set using the anonymized data. 
 
 
     
     
       9. The system of  claim 8 , wherein the data hierarchy comprises a lower level and a higher level, and wherein the query processor further to:
 match the data from the column with a lower level location in the lower level of the data hierarchy; and 
 identify the less specific data in the higher level at a higher level location that relates to the lower level location. 
 
     
     
       10. The system of  claim 8 , wherein the query processor further to:
 determine that the privacy category is an identifier semantic type or a quasi-identifier type; and 
 perform the anonymization of the data in response to the determination that the privacy category is the identifier semantic type or the quasi-identifier type. 
 
     
     
       11. The system of  claim 8 , wherein the query processor further to:
 input the feature vector into a trained machine learning model, wherein the trained machine learning model comprises an encoded threshold and identifies one of the one or more candidate semantic categories having a corresponding probability above the encoded threshold; and 
 assign the candidate semantic category identified by the trained machine learning model to the column. 
 
     
     
       12. The system of  claim 8 , wherein the query processor further to:
 anonymize the data, for each of the plurality of columns on an individual basis, based on the privacy category of the corresponding column. 
 
     
     
       13. The system of  claim 8 , wherein the query processor further to:
 select a column from the plurality of columns; 
 apply the bloom filter with a potential semantic category to the data of the column selected; and 
 compute the probability for the column selected based on a set of results from the applying of the bloom filter to the data in the column selected. 
 
     
     
       14. The system of  claim 13 , wherein the bloom filter is selected from the group consisting of a whitelist bloom filter and a blacklist bloom filter. 
     
     
       15. A non-transitory machine-readable medium storing instructions which, when executed by one or more processors of a computing device, cause the one or more processors to:
 retrieve data from a data set, wherein the data is organized in a plurality of columns; 
 for each column in the plurality of columns:
 generate, with the one or more processors, one or more candidate semantic categories for the column, wherein each of the one or more candidate semantic categories is included in a plurality of semantic categories; 
 generate a probability for each of the one or more candidate semantic categories by applying a bloom filter to the data of the column; 
 create a feature vector for the column from the one or more candidate semantic categories and the corresponding probabilities; 
 determine a semantic category of the column based on the feature vector; 
 determine a privacy category based on a designation of the semantic category; and 
 anonymize the data in the column based on the privacy category to produce anonymized data, wherein the anonymizing comprises replacing more specific data in the column with less specific data based on a data hierarchy of the plurality of semantic categories that relates the more specific data to the less specific data, and wherein the less specific data reduces a likelihood that one or more data subjects are identifiable from the anonymized data; and 
 
 generate an anonymized view of the data set using the anonymized data. 
 
     
     
       16. The non-transitory machine-readable medium of  claim 15 , wherein the data hierarchy comprises a lower level and a higher level, and wherein the instructions further cause the one or more processors to:
 match the data from the column with a lower level location in the lower level of the data hierarchy; and 
 identify the less specific data in the higher level at a higher level location that relates to the lower level location. 
 
     
     
       17. The non-transitory machine-readable medium of  claim 15 , wherein the instructions further cause the one or more processors to:
 determine that the privacy category is an identifier semantic type or a quasi-identifier type; and 
 perform the anonymization of the data in response to the determination that the privacy category is the identifier semantic type or the quasi-identifier type. 
 
     
     
       18. The non-transitory machine-readable medium of  claim 15 , wherein the instructions further cause the one or more processors to:
 input the feature vector into a trained machine learning model, wherein the trained machine learning model comprises an encoded threshold and identifies one of the one or more candidate semantic categories having a corresponding probability above the encoded threshold; and 
 assign the candidate semantic category identified by the trained machine learning model to the column. 
 
     
     
       19. The non-transitory machine-readable medium of  claim 15 , wherein the instructions further cause the one or more processors to:
 anonymize the data, for each of the plurality of columns on an individual basis, based on the privacy category of the corresponding column. 
 
     
     
       20. The non-transitory machine-readable medium of  claim 15 , wherein the instructions further cause the one or more processors to:
 select a column from the plurality of columns; 
 apply the bloom filter with a potential semantic category to the data of the column selected; and 
 compute the probability for the column selected based on a set of results from the applying of the bloom filter to the data in the column selected.