Generating datasets for machine learning systems

Abstract

Disclosed herein are embodiments of systems, methods, and products comprising an analytic server that automates training dataset generation for different application areas. The server may perform an automated, iterative refinement process to build a collection of dataset generator models over time. The server may receive a set of seed examples in a domain and generate candidate examples based on the features of the seed examples using data synthesis techniques. The server may execute a pre-trained label discriminator (LD) and domain discriminator (D2) on the candidate examples. The LD may identify and reject mislabeled data. The D2 may identify and reject out of domain data. The analytic server may regenerate new labeled data based on the feedback of the LD and D2. The analytic server may train a dataset generator by iteratively performing these steps for refinement until the regenerated candidate examples reach a pass rate threshold.

What is claimed is: 
     
         1 . A computer-implemented method for training generative models to generate labeled datasets, the method comprising:
 generating, by a computer, one or more candidate labeled datasets based upon a plurality of dataset features of one or more example datasets associated with a domain;   applying, by the computer, a generative model on the plurality of dataset features to train the generative model to identify a mislabeled dataset of the one or more candidate labeled datasets;   applying, by the computer, the generative model of the machine-learning architecture to train the generative model to identify an unrelated dataset of the one or more candidate labeled datasets that is out of the domain; and   responsive to the computer determining that the generative model satisfies one or more training pass rates based upon a number of mislabeled datasets and a number of unrelated datasets, storing the generative model into a database.   
     
     
         2 . The method according to  claim 1 , wherein identifying the plurality of dataset features includes receiving, by the computer, one or more seed examples in the domain and comprising the one or more example datasets having a limited number of datasets of a same data type. 
     
     
         3 . The method according to  claim 2 , further comprising identifying, by the computer, a plurality of dataset features of the one or more seed examples, wherein the computer generates the one or more candidate labeled datasets using the plurality of dataset features. 
     
     
         4 . The method according to  claim 2 , further comprising generating, by the computer, the one or more candidate labeled datasets by combining the limited number of datasets with a plurality of external datasets obtained from different sources. 
     
     
         5 . The method according to  claim 1 , wherein the generative model includes a label discriminator, wherein the computer trains the label discriminator to identify the mislabeled dataset, and generate a new labeled dataset having an accurate label. 
     
     
         6 . The method according to  claim 5 , wherein the label discriminator includes k-binary classification functions or a k-class classifier having k number of distinct labels. 
     
     
         7 . The method according to  claim 1 , wherein the generative model includes a domain discriminator, wherein the computer trains the domain discriminator to identify the unrelated dataset out of the domain, and generate a new labeled dataset in the domain. 
     
     
         8 . The method according to  claim 7 , wherein the domain discriminator is a binary classifier based on a deep auto-encoder neural network. 
     
     
         9 . The method according to  claim 1 , further comprising:
 receiving, by the computer, a request to generate one or more datasets in the domain;   retrieving, by the computer from the database, the generative model trained for the domain indicated by the request; and   executing, by the computer, the generative model to generate the one or more datasets in the domain.   
     
     
         10 . The method according to  claim 1 , wherein the computer trains the generative model using progressive generative adversarial networks learning algorithms. 
     
     
         11 . A computer system for training and managing generative models that generate labeled datasets, the system comprising:
 a non-transitory storage of a database configured to store one or more generative models trained for corresponding one or more domains; and   a computer in communication with the database and configured to:
 generate one or more candidate labeled datasets based upon a plurality of dataset features of one or more example datasets associated with a domain; 
 apply a generative model on the plurality of dataset features to train the generative model to identify a mislabeled dataset of the one or more candidate labeled datasets; 
 apply the generative model of the machine-learning architecture to train the generative model to identify an unrelated dataset of the one or more candidate labeled datasets that is out of the domain; and 
 responsive to the computer determining that the generative model satisfies one or more training pass rates based upon a number of mislabeled datasets and a number of unrelated datasets, store the generative model into the database. 
   
     
     
         12 . The system according to  claim 11 , wherein when identifying the plurality of dataset features the computer is further configured to receive one or more seed examples in the domain and comprising the one or more example datasets having a limited number of datasets of a same data type. 
     
     
         13 . The system according to  claim 12 , wherein the computer is further configured to identify a plurality of dataset features of the one or more seed examples, wherein the computer generates the one or more candidate labeled datasets using the plurality of dataset features. 
     
     
         14 . The system according to  claim 12 , wherein the computer is further configured to generate the one or more candidate labeled datasets by combining the limited number of datasets with a plurality of external datasets obtained from different sources. 
     
     
         15 . The system according to  claim 11 , wherein the generative model includes a label discriminator, wherein the computer trains the label discriminator to identify the mislabeled dataset, and generate a new labeled dataset having an accurate label. 
     
     
         16 . The system according to  claim 15 , wherein the label discriminator includes k-binary classification functions or a k-class classifier having k number of distinct labels. 
     
     
         17 . The system according to  claim 11 , wherein the generative model includes a domain discriminator, wherein the computer trains the domain discriminator to identify the unrelated dataset out of the domain, and generate a new labeled dataset in the domain. 
     
     
         18 . The system according to  claim 17 , wherein the domain discriminator is a binary classifier based on a deep auto-encoder neural network. 
     
     
         19 . The system according to  claim 11 , wherein the computer is further configured to:
 receive a request to generate one or more datasets in the domain;   retrieve, from the database, the generative model trained for the domain indicated by the request; and   execute the generative model to generate the one or more datasets in the domain.   
     
     
         20 . The system according to  claim 11 , wherein the computer trains the generative model using progressive generative adversarial networks learning algorithms.