Systems and Methods for Improved Adversarial Training of Machine-Learned Models

Abstract

Example aspects of the present disclosure are directed to systems and methods that enable improved adversarial training of machine-learned models. An adversarial training system can generate improved adversarial training examples by optimizing or otherwise tuning one or hyperparameters that guide the process of generating of the adversarial examples. The adversarial training system can determine, solicit, or otherwise obtain a realism score for an adversarial example generated by the system. The realism score can indicate whether the adversarial example appears realistic. The adversarial training system can adjust or otherwise tune the hyperparameters to produce improved adversarial examples (e.g., adversarial examples that are still high-quality and effective while also appearing more realistic). Through creation and use of such improved adversarial examples, a machine-learned model can be trained to be more robust against (e.g., less susceptible to) various adversarial techniques, thereby improving model, device, network, and user security and privacy.

1 - 20 . (canceled) 
     
     
         21 . A computer-implemented method, the method comprising:
 perturbing, by one or more computing devices, image data to generate adversarial image data configured to cause a machine-learned model to misrecognize content depicted in the image data;   providing, by the one or more computing devices to a user device, the adversarial image data for display on the user device;   receiving, by the one or more computing devices and from the user device, user feedback indicating recognition by the user of the content; and   training, by the one or more computing devices, the machine-learned model based at least in part on the adversarial image data.   
     
     
         22 . The method of  claim 21 , wherein the user feedback comprises a class assigned to the content by the user. 
     
     
         23 . The method of  claim 21 , wherein the content corresponds to a class, and wherein the user feedback indicates recognition of the class by the user. 
     
     
         24 . The method of  claim 21 , wherein the user feedback indicates that the content is readable. 
     
     
         25 . The method of  claim 21 , wherein perturbing the image data comprises:
 determining, by the one or more computing devices, a direction of a gradient of a loss function that evaluates an output provided by the machine-learned model when given at least a portion of the image data as an input;   perturbing, by the one or more computing devices, the image data in a second direction that is opposite to the direction of the gradient of the loss function; and   generating, by the one or more computing devices and based at least in part on the perturbed image data, the adversarial image data.   
     
     
         26 . The method of  claim 21 , wherein the image data is perturbed according to one or more updated perturbation parameters, wherein the one or more updated perturbation parameters were obtained by:
 perturbing, by the one or more computing devices, first image data to generate first adversarial image data configured to cause the machine-learned model to misrecognize first content depicted in the first image data, wherein the first image data is perturbed according to one or more first perturbation parameters;   providing, by the one or more computing devices to a first user device, the first adversarial image data for display on the first user device;   receiving, by the one or more computing devices and from the first user device, first user feedback indicating recognition failure by the first of the first content; and   updating, automatically by the one or more computing devices, the one or more first perturbation parameters to obtain the one or more updated perturbation parameters, the one or more updated perturbation parameters configured to decrease a magnitude of the perturbation.   
     
     
         27 . The method of  claim 26 , wherein the one or more first perturbation parameters comprise a step size hyperparameter that controls a magnitude of a step performed during the perturbation. 
     
     
         28 . The method of  claim 27 , wherein updating the one or more first perturbation parameters comprises decreasing, automatically by the one or more computing devices, the step size. 
     
     
         29 . A computing system, comprising:
 one or more processors; and   one or more non-transitory computer-readable media storing instructions that are executable by the one or more processors to cause the computing system to perform operations, the operations comprising:
 perturbing image data to generate adversarial image data configured to cause a machine-learned model to misrecognize content depicted in the image data; 
 providing, to a user device, the adversarial image data for display on the user device; 
 receiving, from the user device, user feedback indicating recognition by the user of the content; and 
 training the machine-learned model based at least in part on the adversarial image data. 
   
     
     
         30 . The computing system of  claim 29 , wherein the user feedback comprises a class assigned to the content by the user. 
     
     
         31 . The computing system of  claim 29 , wherein the content corresponds to a class, and wherein the user feedback indicates recognition of the class by the user. 
     
     
         32 . The computing system of  claim 29 , wherein the user feedback indicates that the content is readable. 
     
     
         33 . The computing system of  claim 29 , wherein perturbing the image data comprises:
 determining a direction of a gradient of a loss function that evaluates an output provided by the machine-learned model when given at least a portion of the image data as an input;   perturbing the image data in a second direction that is opposite to the direction of the gradient of the loss function; and   generating, based at least in part on the perturbed image data, the adversarial image data.   
     
     
         34 . The computing system of  claim 29 , wherein the image data is perturbed according to one or more updated perturbation parameters, wherein the one or more updated perturbation parameters were obtained by:
 perturbing first image data to generate first adversarial image data configured to cause the machine-learned model to misrecognize first content depicted in the first image data, wherein the first image data is perturbed according to one or more first perturbation parameters;   providing, to a first user device, the first adversarial image data for display on the first user device;   receiving, from the first user device, first user feedback indicating recognition failure by the first of the first content; and   updating, automatically, the one or more first perturbation parameters to obtain the one or more updated perturbation parameters, the one or more updated perturbation parameters configured to decrease a magnitude of the perturbation.   
     
     
         35 . The computing system of  claim 34 , wherein the one or more first perturbation parameters comprise a step size hyperparameter that controls a magnitude of a step performed during the perturbation. 
     
     
         36 . The computing system of  claim 35 , wherein updating the one or more first perturbation parameters comprises decreasing, automatically by the one or more computing devices, the step size. 
     
     
         37 . One or more non-transitory computer-readable media storing instructions that are executable by one or more processors to cause a computing system to perform operations, the operations comprising:
 perturbing image data to generate adversarial image data configured to cause a machine-learned model to misrecognize content depicted in the image data;   providing, to a user device, the adversarial image data for display on the user device;   receiving, from the user device, user feedback indicating recognition by the user of the content; and   training the machine-learned model based at least in part on the adversarial image data.   
     
     
         38 . The one or more non-transitory computer-readable media of  claim 37 , wherein the user feedback comprises a class assigned to the content by the user. 
     
     
         39 . The one or more non-transitory computer-readable media of  claim 37 , wherein the content corresponds to a class, and wherein the user feedback indicates recognition of the class by the user. 
     
     
         40 . The one or more non-transitory computer-readable media of  claim 37 , wherein the user feedback indicates that the content is readable.