Systems and techniques to monitor text data quality

Abstract

Disclosed are a system, apparatus and techniques for evaluating a dataset to confirm that the data in the dataset satisfies a data quality metric. A machine learning engine or the like may evaluate text strings within the dataset may be of arbitrary length and encoded according to an encoding standard. Data vectors of a preset length may be generated from the evaluated text strings using various techniques. Each data vector may be representative of the content of the text string and a category may be assigned to the respective data vector. The category assigned to each data vectors may be evaluated with respect to other data vectors in the dataset to determine compliance with a quality metric. In the case that a number of data vectors fail to meet a predetermined quality metric, an alert may be generated to mitigate any system errors that may result from unsatisfactory data quality.

What is claimed is: 
     
       1. A method, comprising:
 identifying a plurality of text strings of alphanumeric characters in a dataset, wherein each respective text string has an arbitrary length and is encoded according to an encoding standard; 
 generating a data vector for each respective text string, wherein the generated data vector has a preset data length and includes feature data indicating features of the respective text string; 
 determining a respective data vector category for each respective data vector, wherein a data vector category is based on the feature data included in each respective data vector; 
 assigning each respective data vector to the respective data vector category determined for the respective data vector; 
 evaluating the dataset based on each respective data vector category; 
 in response to a result of the evaluation of the dataset, determining whether the dataset satisfies a data quality metric; 
 in response to the data set failing to satisfy the data quality metric, generating an alarm; and 
 reporting how the dataset failed to satisfy the data quality metric and provide a link to the dataset that caused the alarm to be generated. 
 
     
     
       2. The method of  claim 1 , wherein generating a data vector for each respective text string, comprises:
 for each respective text string:
 applying a statistical function to the respective text string to conform the text string to the preset data length; 
 identify, by applying a filter to the respective text string, features of the alphanumeric characters in the respective text string based on the encoding of the respective text string according to the encoding standard; 
 applying weightings to the identified features of the alphanumeric characters in each respective text string based on a position of each feature within the respective text string; 
 extracting feature data based on an output of the applied filter and applied weightings; and 
 generating the data vector for the respective text string by populating the data vector with the extracted feature data of the respective text string. 
 
 
     
     
       3. The method of  claim 2 , wherein the applied statistical function includes at least one of a mean function, a minimization function, or a maximization function. 
     
     
       4. The method of  claim 2 , wherein the applied filter includes at least one of a convolution filter or a triplet filter windowing function. 
     
     
       5. The method of  claim 1 , wherein determining whether the dataset satisfies the data quality metric comprises:
 determining that a percentage of data vectors assigned to a particular data vector category exceeded a predetermined percentage threshold. 
 
     
     
       6. The method of  claim 5 , further comprising:
 in response to the dataset failing to satisfy the data quality metric, generating an alert, wherein the data quality metric is a predetermined outlier threshold value for a number of data vectors being assigned to a data vector category indicating a data vector is an outlier; and 
 forwarding the generated alert to a client device for corrective action, 
 wherein the alert is generated in response to the number of data vectors that are assigned to a data vector category indicating a data vector is an outlier exceeds the predetermined outlier threshold value. 
 
     
     
       7. The method of  claim 1 , wherein determining a respective category for each respective data vector comprises:
 accessing a collection of data vector category definitions, wherein each data vector category definition includes a plurality of features that define the data vector category; and 
 for each respective data vector;
 comparing the feature data of the respective data vector to at least one data vector category definition in the collection of data vector category definitions; 
 based on the comparison, determining a similarity rating between the feature data of the respective data vector and the at least one data vector category definition in the collection; and 
 selecting the at least one data vector category definition in the collection that produced a highest determined similarity rating. 
 
 
     
     
       8. The method of  claim 1 , wherein determining a respective category for each respective data vector comprises:
 processing the respective data vectors in the dataset with respect to one another using a machine learning algorithm; 
 based on an output of the machine learning algorithm, categorizing one or more respective data vectors as outliers, wherein a respective data vector is categorized as an outlier when the feature data of the respective data is substantially different based on a similarity threshold from the feature data of a majority of other respective data vectors in the dataset; and 
 after categorizing the one or more respective data vectors of the dataset as outliers, categorizing the other respective data vectors as normal. 
 
     
     
       9. The method of  claim 1 , wherein determining a respective category for each respective data vector comprises:
 processing the respective data vectors in the dataset with respect to one another using a machine learning algorithm, wherein the machine learning algorithm is a clustering algorithm; and 
 for each respective data vector:
 categorizing the respective data vector based on a cluster in which the clustering algorithm indicated the respective data vector belonged. 
 
 
     
     
       10. A system, comprising:
 a data source, wherein the data source outputs a dataset related to a service and the dataset includes a plurality of text strings of arbitrary length and that are encoded according to an encoding standard; 
 at least one database coupled to the data source and operable to store the dataset output by the data source; and 
 a data quality monitoring component coupled to the data source and operable to perform functions, including functions to:
 receive the dataset including the plurality of text strings of arbitrary length from the data source; 
 generate a data vector for each respective text string of the plurality of text strings, wherein the generated data vector has a preset data length and includes feature data indicating features of the respective text string; 
 determine a respective data vector category for each respective data vector, wherein a category is based on the feature data included in each respective data vector; 
 evaluate the dataset based on each respective data vector category; 
 
 in response to evaluating the dataset, determine whether the dataset satisfies a data quality metric; 
 in response to the data set failing to satisfy the data quality metric, generate an alarm; and 
 report how the dataset failed to satisfy the data quality metric and provide a link to the dataset that caused the alarm to be generated. 
 
     
     
       11. The system of  claim 10 , wherein the data quality monitoring component is further operable to perform functions, including functions to:
 assign each respective data vector to the respective category determined for the respective data vector. 
 
     
     
       12. The system of  claim 10 , wherein the data quality monitoring component further comprises a data vector component, the data vector component operable to perform functions to:
 for each respective text string:
 process the respective text string to adapt the text string to the preset data length; 
 identify features of alphanumeric characters in the respective text string based on the encoding of the respective text string; 
 applying weightings to the identified features based on a position of each feature within the respective text string; 
 extracting feature data based on the identified features and applied weightings; and 
 generating the data vector for the respective text string by populating the data vector with the extracted feature data of the respective text string. 
 
 
     
     
       13. The system of  claim 12 , wherein the data vector component, when identifying features of the alphanumeric characters in the respective text string, is further operable to perform functions to:
 identify occurrences in the text string where a letter is adjacent to a number, a number is adjacent to a letter, a letter is surrounded by letters of a different type, triplets of a data element of the same type, or two numbers or letters of the same type are adjacent to letters or numbers of a different type,
 wherein a type of letter or number is an upper-case letter, a lower-case letter, a Latin character, or an Arabic character. 
 
 
     
     
       14. The system of  claim 12 , wherein the data vector component, when identifying features of the alphanumeric characters in the respective text string, is further operable to perform functions to:
 identify a feature based on locating edges between letters, a group of multiple numbers in serial, or a group of letters in serial. 
 
     
     
       15. A non-transitory computer readable medium embodying programming code that when executed by a processor causes the processor to perform functions, including functions to:
 receive a dataset with a plurality of variable length character strings; 
 for each respective variable length character string in the plurality of variable length character strings:
 compute a plurality of features of the respective variable length character string; 
 capture attributes of the respective variable length character string based on the plurality of computed features; 
 populate a data vector with the captured attributes, wherein the data vector has a predetermined length and includes one or more of the captured attributes of the respective variable length character string; 
 
 assign a category to each respective data vector using a machine learning algorithm; 
 based on the category assigned to each respective data vector, evaluate the dataset; and 
 in response to evaluating the dataset based on the category assigned to each respective data vector, determine whether the dataset satisfies a data quality metric. 
 
     
     
       16. The non-transitory computer readable medium of  claim 15 , further comprising programming code that when executed by a processor causes the processor to perform functions, including functions to:
 in response to a determination that the dataset fails to satisfy the data quality metric, generate an alarm; and device 
 forward the generated alarm to a client with a report indicating how the dataset failed to satisfy the data quality metric and a link to the dataset. 
 
     
     
       17. The non-transitory computer readable medium of  claim 15 , wherein:
 a category is at least one of an upper-case letter, a lower-case letter, a punctuation mark, a Roman letter, an Arabic letter, a number, a punctuation mark, a special character, a space, or an outlier, and 
 the computed features include at least one of a percentage of elements in the character string are in a specific category, a number of edges between letters, a group of numbers in serial, or a group of letters in serial. 
 
     
     
       18. The non-transitory computer readable medium of  claim 15 , wherein an edge is determined to be located between an end of a first character and a start of a second character. 
     
     
       19. The non-transitory computer readable medium of  claim 15 , wherein a captured attribute is further identified as a letter adjacent to a number, a number adjacent to a letter, a letter surrounded by characters of a different category, or triplets of a character of the same category, two characters of the same category adjacent to characters of a different category. 
     
     
       20. The non-transitory computer readable medium of  claim 15 , wherein the data quality metric is one of:
 the dataset includes a percentage of data vectors assigned to an outlier category that is greater than an outlier threshold percentage, or 
 the dataset fails to exceed a number of data vectors assigned to a specific category that corresponds to a data source that provided the dataset.