Efficient data processing to identify information and reformant data files, and applications thereof

Abstract

The present disclosure is directed to systems and methods for identifying demographic information in a data file. The method may include: receiving the data file containing a plurality of fields of demographic information from a third-party, the data file having inconsistent or mislabeled nomenclatures for one or more fields of the plurality of fields or spurious demographic information; analyzing the data file using a machine learning model trained according to other data files to distinguish between each of the plurality of fields of demographic information, the machine learning model being based on a plurality of machine learning algorithms to identify different types demographic information; generating a score indicating a probability that each of the plurality of fields of demographic information was identified correctly; and generating a revised data file labeling each of the plurality of fields of demographic information based on the identified type.

What is claimed is: 
     
         1 . A computer-implemented method of identifying demographic information in a data file, comprising:
 receiving the data file containing a plurality of fields of demographic information from a third-party, the data file having inconsistent or mislabeled nomenclatures for one or more fields of the plurality of fields or spurious demographic information;   analyzing the data file using a machine learning model trained according to other data files to distinguish between each of the plurality of fields of demographic information, the machine learning model being based on a plurality of machine learning algorithms to identify different types demographic information;   generating a score indicating a probability that each of the plurality of fields of demographic information was identified correctly; and   generating a revised data file labeling each of the plurality of fields of demographic information based on the identified type.   
     
     
         2 . The method of  claim 1 , wherein analyzing the data file comprises analyzing semantic content of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         3 . The method of  claim 1 , wherein analyzing the data file comprises analyzing a shape of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         4 . The method of  claim 1 , wherein analyzing the data file comprises analyzing metadata of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         5 . The method of  claim 4 , wherein the metadata includes each nomenclature of each of the plurality of fields of demographic information. 
     
     
         6 . The method of  claim 1 , wherein, in response to identifying different ones of the plurality of fields of demographic information, the method further comprises cross-checking at least one of the plurality of fields of demographic information against known demographic information. 
     
     
         7 . The method of  claim 1 , further comprising transmitting the revised data file to the third-party. 
     
     
         8 . A system for identifying demographic information in a data file, comprising:
 a memory that stores instructions for identifying the demographic information in the data file; and   a processor configured to execute the instructions that cause the processor to:
 receive the data file containing a plurality of fields of demographic information from a third-party, the data file having inconsistent or mislabeled nomenclatures for one or more fields of the plurality of fields or spurious demographic information; 
 analyze the data file using a machine learning model trained according to other data files to distinguish between each of the plurality of fields of demographic information, the machine learning model being based on a plurality of machine learning algorithms to identify different types demographic information; 
 generate a score indicating a probability that each of the plurality of fields of demographic information was identified correctly; and 
 generate a revised data file labeling each of the plurality of fields of demographic information based on the identified type. 
   
     
     
         9 . The system of  claim 8 , wherein analyzing the data file comprises analyzing semantic content of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         10 . The system of  claim 8 , wherein analyzing the data file comprises analyzing a shape of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         11 . The system of  claim 10 , wherein the metadata includes each nomenclature of each of the plurality of fields of demographic information. 
     
     
         12 . The system of  claim 8 , wherein analyzing the data file comprises analyzing each nomenclature to identify the different types of demographic information. 
     
     
         13 . The system of  claim 8 , wherein, in response to identifying different ones of the plurality of fields of demographic information, the instructions further cause the processor to cross-check at least one of the plurality of fields of demographic information against known demographic information. 
     
     
         14 . The system of  claim 8 , wherein the instructions further cause the processor to transmit the revised data file to the third-party. 
     
     
         15 . non-transitory program storage device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform a method, the method comprising:
 receiving the data file containing a plurality of fields of demographic information from a third-party, the data file having inconsistent or mislabeled nomenclatures for one or more fields of the plurality of fields or spurious demographic information;   analyzing the data file using a machine learning model trained according to other data files to distinguish between each of the plurality of fields of demographic information, the machine learning model being based on a plurality of machine learning algorithms to identify different types demographic information;   generating a score indicating a probability that each of the plurality of fields of demographic information was identified correctly; and   generating a revised data file labeling each of the plurality of fields of demographic information based on the identified type.   
     
     
         16 . The method of  claim 15 , wherein analyzing the data file comprises analyzing semantic content of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         17 . The method of  claim 15 , wherein analyzing the data file comprises analyzing a shape of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         18 . The method of  claim 15 , wherein analyzing the data file comprises analyzing metadata of each of the plurality of fields of demographic information to identify the different types of demographic information. 
     
     
         19 . The method of  claim 18 , wherein the metadata includes each nomenclature of each of the plurality of fields of demographic information. 
     
     
         20 . The method of  claim 15 , wherein, in response to identifying different ones of the plurality of fields of demographic information, the method further comprises cross-checking at least one of the plurality of fields of demographic information against known demographic information.