Post-hoc improvement of instance-level and group-level prediction metrics

Abstract

A post-processing method, system, and computer program product for post-hoc improvement of instance-level and group-level prediction metrics, including training a bias detector on a payload data that learns to detect a sample in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, suggesting, in the run-time, a de-biased prediction based on the selected biased sample by a de-biasing procedure, and an arbiter decides based on user feedback whether to use the de-biased prediction or an original prediction made prior to the de-biasing procedure from the customer model which is then used as an output.

What is claimed is: 
     
         1 . A post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method comprising:
 training a bias detector on a payload data that learns to detect a sample in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, wherein, during the training:
 the bias detector perturbs a protected attribute in the payload data and computes the individual bias as an individual bias score by iteratively finding a difference between a probability of a favorable outcome for the perturbed protected attribute to original data of the payload data; 
   applying, in a run-time, the bias detector on a run-time sample to select a biased sample in the run-time sample having an individual bias greater than the predetermined individual bias threshold value; and   suggesting, in the run-time, a de-biased prediction based on the selected biased sample by a de-biasing procedure:
 perturbing the protected attribute in the payload data; 
 running the perturbed examples through the customer model; and 
 picking a most likely prediction for the perturbed data as a suggested value to modify, 
   wherein an arbiter decides based on user feedback whether to use the de-biased prediction or an original prediction made prior to the de-biasing procedure from the customer model which is then used as an output.   
     
     
         2 . The post-processing computer-implemented method of  claim 1 , wherein the applying and the suggesting operate in a post-processing that targets the sample with the individual value for remediation in order to change the value of the sample based both on individual and group fairness metrics. 
     
     
         3 . The post-processing computer-implemented method of  claim 1 , wherein the trained detector is received as a black-box. 
     
     
         4 . The post-processing computer-implemented method of  claim 1 , wherein the applying comprises a pure run-time approach that does not require ground truth class labels for a validation set. 
     
     
         5 . The post-processing computer-implemented method of  claim 1 , wherein the group bias is of a group that includes the sample. 
     
     
         6 . The post-processing computer-implemented method of  claim 1 , wherein, during the applying in the run-time, the bias detector perturbs the protected attribute in the payload data for unprivileged group samples out of the group, and computes individual bias scores for each of the unprivileged group samples by finding a difference between a probability of favorable outcomes for the perturbed unprivileged group samples and the original prediction. 
     
     
         7 . The post-processing computer-implemented method of  claim 1 , wherein, in a case of multiple perturbations including a plurality of unprivileged group samples out of the group, an average probability of favorable outcomes is used as the probability of favorable outcomes for the perturbed unprivileged group samples and the original prediction. 
     
     
         8 . A post-processing computer-implemented method for post-hoc improvement of instance-level and group-level prediction metrics, the post-processing method comprising:
 testing a plurality of samples from an unprivileged group for an individual bias in a customer model; and   assigning an outcome of a privileged group to a sample of the plurality of samples based on the sample being tested and the sample having a bias greater than a predetermined individual bias threshold value,   wherein the testing is performed using a trained bias detector that is trained on a payload data and that learns to detect a sample of the plurality of samples in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, the sample being a member of an unprivileged group,   suggesting, in the run-time, a de-biased prediction for only a biased sample that is the member of the unprivileged group by a de-biasing procedure for each sample point by:
 perturbing the protected attribute in the payload data; 
 running the perturbed examples through the customer model; and 
 picking a most likely prediction for the perturbed data as a suggested value to modify, 
   wherein an arbiter decides based on user feedback whether to use the de-biased prediction or an original prediction made prior to the de-biasing procedure from the customer model which is then used as an output.   
     
     
         9 . The post-processing computer-implemented method of  claim 8 , wherein the unprivileged group received the outcome based on the individual bias according to a classification of the unprivileged group. 
     
     
         10 . The post-processing computer-implemented method of  claim 8 , wherein the outcome of samples in the privileged group are unchanged by the assigning. 
     
     
         11 . A post-processing computer program product for post-hoc improvement of instance-level and group-level prediction metrics, the computer program product comprising a computer-readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:
 training a bias detector on a payload data that learns to detect a sample in a customer model that has an individual bias greater than a predetermined individual bias threshold value with constraints on a group bias, wherein, during the training:
 the bias detector perturbs a protected attribute in the payload data and computes the individual bias as an individual bias score by iteratively finding a difference between a probability of a favorable outcome for the perturbed protected attribute to original data of the payload data; 
   applying, in a run-time, the bias detector on a run-time sample to select a biased sample in the run-time sample having an individual bias greater than the predetermined individual bias threshold value; and   suggesting, in the run-time, a de-biased prediction based on the selected biased sample by a de-biasing procedure:
 perturbing the protected attribute in the payload data; 
 running the perturbed examples through the customer model; and 
 picking a most likely prediction for the perturbed data as a suggested value to modify, 
   wherein an arbiter decides based on user feedback whether to use the de-biased prediction or an original prediction made prior to the de-biasing procedure from the customer model which is then used as an output.