Counterfactual based gender fairness algorithm for large language models

Abstract

Gender bias assessment techniques that are available currently only consider two genders viz male and female and is not inclusive of lesbian, gay, bisexual, transgender and queer (LGBTQ) genders. Present disclosure provides method and system for assessing gender fairness of large language models (LLM). The system first receive context aware prompt comprising context and prompt and then converts names present in the context using name anonymization technique. The system then convert many-to-many pronouns present in no-name-context into many to one pronouns which are then used to create counterfactual context for each gender pronoun group. Thereafter, system queries LLM with counterfactual prompt created using counterfactual context to obtain counterfactual response which is then used to create embedding data frame. Further, system trains Gaussian Mixture Model using embedding data frame which is then utilized to calculate cluster distances. Finally, system assess gender bias based on cluster distances and predefined distance threshold.

What is claimed is: 
     
         1 . A processor implemented method, comprising:
 receiving, by a system via one or more hardware processors, a context aware prompt, the context aware prompt comprising a context and a prompt;   masking, by the system via the one or more hardware processors, one or more names present in the context using a name anonymization technique to obtain a no-name-context, wherein the no-name-context comprises a plurality of primary many-to-one pronouns and a plurality of many-to-many pronouns;   converting, by the system via the one or more hardware processors, the plurality of many-to-many pronouns present in the no-name-context into a plurality of secondary many-to-one pronouns, wherein an intermediate counterfactual context is obtained after the conversion, wherein the intermediate counterfactual context comprises a plurality of many-to-one pronouns, and wherein the plurality of many-to-one pronouns is a combination of the plurality of primary many-to-one pronouns and the plurality of secondary many-to-one pronouns;   identifying, by the system via the one or more hardware processors, each many to one pronoun of the plurality of many-to-one pronouns and one or more honorific titles that are present in the intermediate counterfactual context using a first custom pattern finder, wherein the first custom pattern finder creates a first custom pattern of the plurality of many to one pronouns and the one or more honorific titles in which each many to one pronoun and each honorific title is separated by an operator;   predicting, by the system via the one or more hardware processors, a pronoun classification for each many-to-one pronoun of the plurality of many-to-one pronouns and each honorific title of the one or more honorific titles present in the first custom pattern created for the intermediate counterfactual context using a pre-trained decision tree classifier, wherein each many-to-one pronoun and each honorific title is replaced with a corresponding predicted pronoun classification in the intermediate counterfactual context to obtain a pronoun classification context;   creating, by the system via the one or more hardware processors, a counterfactual context for each gender pronoun group of a plurality of predefined gender pronoun groups based on the pronoun classification context;   inserting, by the system via the one or more hardware processors, the counterfactual context generated for each gender pronoun group into the prompt to obtain a counterfactual prompt for each gender pronoun group;   querying, by the system via the one or more hardware processors, a large language model (LLM) with the counterfactual prompt obtained for each gender pronoun group to obtain a counterfactual response corresponding to each gender pronoun group, wherein a plurality of counterfactual responses are obtained corresponding to a plurality of predefined gender pronoun groups;   generating, by the system via the one or more hardware processors, one or more counterfactual response embeddings for each counterfactual response of the plurality of counterfactual responses using one or more sentence embedding models, wherein a plurality of counterfactual response embeddings are generated by each sentence embedding model of the one or more sentence embedding models;   for each sentence embedding model of the one or more sentence embedding models, creating, by the system via the one or more hardware processors, a sentence embedding model based cluster distance list by performing:
 creating, by the system via the one or more hardware processors, an embedding data frame, wherein the embedding data frame of a sentence embedding model comprises the plurality of counterfactual response embeddings generated for the respective sentence embedding model; 
 training, by the system via the one or more hardware processors, a Gaussian Mixture Model for a single cluster using the created embedding data frame of the respective sentence embedding model; 
 determining, by the system via the one or more hardware processors, a cluster center of the single cluster; 
 calculating, by the system via the one or more hardware processors, a cluster distance of each counterfactual response embedding of the plurality of counterfactual response embeddings from the cluster center using a Euclidean distance; and 
 adding, by the system via the one or more hardware processors, the cluster distance calculated for each counterfactual response embedding to the sentence embedding model based cluster distance list predefined for the corresponding sentence embedding model, wherein the sentence embedding model based cluster distance list obtained after addition of cluster distances comprises a plurality of calculated cluster distances in form of a plurality of elements; 
   determining, by the system via the one or more hardware processors, whether any element among the plurality of elements present in either sentence embedding model based cluster distance list is more than a predefined distance threshold; and   displaying, by the system via the one or more hardware processors, a gender bias result based on the determination.   
     
     
         2 . The processor implemented method of  claim 1 , wherein displaying the gender bias result comprises:
 displaying the gender bias result as ‘LLM is unfair’ upon determining that at least one element in the plurality of elements present in either sentence embedding model based cluster distance list is more than the predefined distance threshold.   
     
     
         3 . The processor implemented method of  claim 1 , wherein displaying the gender bias result further comprises:
 displaying the gender bias result as ‘LLM is fair’ upon determining that no element in the plurality of elements present in either sentence embedding model based cluster distance list is more than the predefined distance threshold.   
     
     
         4 . The processor implemented method of  claim 1 , wherein conversion of the plurality of many-to-many pronouns present in the no-name-context into the plurality of secondary many-to-one pronouns comprises:
 identifying, by the system via the one or more hardware processors, one or more sentences containing the plurality of many-to-many pronouns in the no-name-context using a second custom pattern finder, wherein the second custom pattern finder creates a second custom pattern that identifies the one or more sentences containing the plurality of many-to-many pronouns; and   converting, by the system via the one or more hardware processors, each identified many-to-many pronoun to a corresponding pronoun in an unambiguous pronoun group using a custom prompt to the LLM, wherein conversion of each identified many-to-many pronoun to the corresponding pronoun in the unambiguous pronoun group generates the plurality of secondary many-to-one pronouns.   
     
     
         5 . The processor implemented method of  claim 1 , wherein the step of creating the counterfactual context for each gender pronoun group of the plurality of predefined gender pronoun groups based on the pronoun classification context comprises:
 identifying, by the system via the one or more hardware processors, a plurality of pronoun classifications present in the pronoun classification context using a third custom pattern finder;   mapping, by the system via the one or more hardware processors, each identified pronoun classification with a corresponding counterfactual gender pronoun, wherein a plurality of mappings are obtained corresponding to the plurality of pronoun classifications;   appending, by the system via the one or more hardware processors, the plurality of mappings to the one or more honorific titles;   creating, by the system via the one or more hardware processors, a third custom pattern comprising a pronoun classification of each of the plurality of mappings, wherein each mapping is separated by the operator;   creating, by the system via the one or more hardware processors, a lambda function that fetches a corresponding gender pronoun for each of the pronoun classifications, wherein the lambda function is utilized in a custom pattern substitution function; and   creating, by the system via the one or more hardware processors, the counterfactual context for each gender by using the custom pattern substitution function based on the mapping of the pronoun classifications with corresponding counterfactual gender pronouns and honorific titles in the pronoun classification context.   
     
     
         6 . A system, comprising:
 a memory storing instructions;   one or more communication interfaces; and   one or more hardware processors coupled to the memory via the one or more communication interfaces, wherein the one or more hardware processors are configured by the instructions to:   receive the context aware prompt comprising a context and a prompt;   mask one or more names present in the context using a name anonymization technique to obtain a no-name-context, wherein the no-name-context comprises a plurality of primary many-to-one pronouns and a plurality of many-to-many pronouns;   convert wherein an intermediate counterfactual context is obtained after the conversion, wherein the intermediate counterfactual context comprises a plurality of many-to-one pronouns, and wherein the plurality of many-to-one pronouns is a combination of the plurality of primary many-to-one pronouns and the plurality of secondary many-to-one pronouns;   identify each many to one pronoun of the plurality of many-to-one pronouns and one or more honorific titles that are present in the intermediate counterfactual context using a first custom pattern finder, wherein the first custom pattern finder creates a first custom pattern of the plurality of many to one pronouns and the one or more honorific titles in which each many to one pronoun and each honorific title is separated by an operator;   predict a pronoun classification for each identified many-to-one pronoun of the plurality of many-to-one pronouns and each honorific title of the one or more honorific titles present in the first custom pattern created for the intermediate counterfactual context using a pre-trained decision tree classifier, wherein each many-to-one pronoun and honorific title is replaced with a corresponding predicted pronoun classification in the intermediate counterfactual context to obtain a pronoun classification context;   create a counterfactual context for each gender pronoun group of a plurality of predefined gender pronoun groups based on the pronoun classification context;   insert the counterfactual context generated for each gender pronoun group into the prompt to obtain a counterfactual prompt for each gender pronoun group;   query a large language model (LLM) with the counterfactual prompt obtained for each gender pronoun group to obtain a counterfactual response corresponding to each gender pronoun group, wherein a plurality of counterfactual responses are obtained corresponding to a plurality of predefined gender pronoun groups;   generate one or more counterfactual response embeddings for each counterfactual response of the plurality of counterfactual responses using one or more sentence embedding models, wherein a plurality of counterfactual response embeddings are generated by each sentence embedding model of the one or more sentence embedding models;   for each sentence embedding model of the one or more sentence embedding models, create a sentence embedding model based cluster distance list by performing:
 create an embedding data frame, wherein the embedding data frame of a sentence embedding model comprises the plurality of counterfactual response embeddings generated for the respective sentence embedding model; 
 train Gaussian Mixture Model for a single cluster using the created embedding data frame of the respective sentence embedding model; 
 determine a cluster center of the single cluster; 
 calculate a cluster distance of each counterfactual response embedding of the plurality of counterfactual response embeddings from the cluster center using a Euclidean distance; 
 add the cluster distance calculated for each counterfactual response embedding to the sentence embedding model based cluster distance list predefined for the corresponding sentence embedding model, wherein the sentence embedding model based cluster distance list obtained after addition of cluster distances comprises a plurality of calculated cluster distances in form of a plurality of elements; 
   determine whether any element among the plurality of elements present in either sentence embedding model based cluster distance list is more than a predefined distance threshold; and   display a gender bias result based on the determination.   
     
     
         7 . The system of  claim 6 , wherein for displaying the gender bias result, the one or more hardware processors ( 204 ) are configured by the instructions to:
 display the gender bias result as ‘LLM is unfair’ upon determining that at least one element in the plurality of elements present in either sentence embedding model based cluster distance list is more than the predefined distance threshold.   
     
     
         8 . The system of  claim 6 , wherein for displaying the gender bias result, the one or more hardware processors ( 204 ) are further configured by the instructions to:
 display the gender bias result as ‘LLM is fair’ upon determining that no element in the plurality of elements present in either sentence embedding model based cluster distance list is more than the predefined distance threshold.   
     
     
         9 . The system of  claim 6 , wherein for converting of the plurality of many-to-many pronouns present in the no-name-context into the plurality of secondary many-to-one pronouns, the one or more hardware processors ( 204 ) are further configured by the instructions to:
 identify one or more sentences containing the plurality of many-to-many pronouns in the no-name-context using a second custom pattern finder, wherein the second custom pattern finder creates a second custom pattern that identifies the one or more sentences containing the plurality of many-to-many pronouns; and   convert each identified many-to-many pronoun to a corresponding pronoun in an unambiguous pronoun group using a custom prompt to the LLM, wherein conversion of each identified many-to-many pronoun to the corresponding pronoun in the unambiguous pronoun group generates the plurality of secondary many-to-one pronouns.   
     
     
         10 . The system of  claim 6 , wherein for creating the counterfactual context for each gender pronoun group of the plurality of predefined gender pronoun groups based on the pronoun classification context, the one or more hardware processors are further configured by the instructions to:
 identify a plurality of pronoun classifications present in the pronoun classification context using a third custom pattern finder;   map each identified pronoun classification with a corresponding counterfactual gender pronoun, wherein a plurality of mappings are obtained corresponding to the plurality of pronoun classifications;   append the plurality of mappings to the one or more honorific titles;   create a third custom pattern comprising a pronoun classification of each of the plurality of mappings, wherein each mapping is separated by an operator;   create a lambda function that fetches a corresponding gender pronoun for each of the pronoun classifications, wherein the lambda function is utilized in a custom pattern substitution function; and   create the counterfactual context for each gender by using the custom pattern substitution function based on the mapping of the pronoun classifications with corresponding counterfactual gender pronouns in the pronoun classification context.   
     
     
         11 . One or more non-transitory machine-readable information storage mediums comprising one or more instructions which when executed by one or more hardware processors cause:
 receiving a context aware prompt, the context aware prompt comprising a context and a prompt;   masking one or more names present in the context using a name anonymization technique to obtain a no-name-context, wherein the no-name-context comprises a plurality of primary many-to-one pronouns and a plurality of many-to-many pronouns;   converting the plurality of many-to-many pronouns present in the no-name-context into a plurality of secondary many-to-one pronouns, wherein an intermediate counterfactual context is obtained after the conversion, wherein the intermediate counterfactual context comprises a plurality of many-to-one pronouns, and wherein the plurality of many-to-one pronouns is a combination of the plurality of primary many-to-one pronouns and the plurality of secondary many-to-one pronouns;   identifying each many to one pronoun of the plurality of many-to-one pronouns and one or more honorific titles that are present in the intermediate counterfactual context using a first custom pattern finder, wherein the first custom pattern finder creates a first custom pattern of the plurality of many to one pronouns and the one or more honorific titles in which each many to one pronoun and each honorific title is separated by an operator;   predicting a pronoun classification for each many-to-one pronoun of the plurality of many-to-one pronouns and each honorific title of the one or more honorific titles present in the first custom pattern created for the intermediate counterfactual context using a pre-trained decision tree classifier, wherein each many-to-one pronoun and each honorific title is replaced with a corresponding predicted pronoun classification in the intermediate counterfactual context to obtain a pronoun classification context;   creating a counterfactual context for each gender pronoun group of a plurality of predefined gender pronoun groups based on the pronoun classification context;   inserting the counterfactual context generated for each gender pronoun group into the prompt to obtain a counterfactual prompt for each gender pronoun group;   querying a large language model (LLM) with the counterfactual prompt obtained for each gender pronoun group to obtain a counterfactual response corresponding to each gender pronoun group, wherein a plurality of counterfactual responses are obtained corresponding to a plurality of predefined gender pronoun groups;   generating one or more counterfactual response embeddings for each counterfactual response of the plurality of counterfactual responses using one or more sentence embedding models, wherein a plurality of counterfactual response embeddings are generated by each sentence embedding model of the one or more sentence embedding models;   for each sentence embedding model of the one or more sentence embedding models, creating a sentence embedding model based cluster distance list by performing:
 creating an embedding data frame, wherein the embedding data frame of a sentence embedding model comprises the plurality of counterfactual response embeddings generated for the respective sentence embedding model; 
 training a Gaussian Mixture Model for a single cluster using the created embedding data frame of the respective sentence embedding model; 
 determining a cluster center of the single cluster; 
 calculating a cluster distance of each counterfactual response embedding of the plurality of counterfactual response embeddings from the cluster center using a Euclidean distance; and 
 adding the cluster distance calculated for each counterfactual response embedding to the sentence embedding model based cluster distance list predefined for the corresponding sentence embedding model, wherein the sentence embedding model based cluster distance list obtained after addition of cluster distances comprises a plurality of calculated cluster distances in form of a plurality of elements; 
   determining whether any element among the plurality of elements present in either sentence embedding model based cluster distance list is more than a predefined distance threshold; and   displaying a gender bias result based on the determination.   
     
     
         12 . The one or more non-transitory machine readable information storage mediums of  claim 11 , wherein displaying the gender bias result comprises:
 displaying the gender bias result as ‘LLM is unfair’ upon determining that at least one element in the plurality of elements present in either sentence embedding model based cluster distance list is more than the predefined distance threshold.   
     
     
         13 . The one or more non-transitory machine readable information storage mediums of  claim 11 , wherein displaying the gender bias result further comprises:
 displaying the gender bias result as ‘LLM is fair’ upon determining that no element in the plurality of elements present in either sentence embedding model based cluster distance list is more than the predefined distance threshold.   
     
     
         14 . The one or more non-transitory machine readable information storage mediums of  claim 11 , wherein conversion of the plurality of many-to-many pronouns present in the no-name-context into the plurality of secondary many-to-one pronouns comprises:
 identifying one or more sentences containing the plurality of many-to-many pronouns in the no-name-context using a second custom pattern finder, wherein the second custom pattern finder creates a second custom pattern that identifies the one or more sentences containing the plurality of many-to-many pronouns; and   converting each identified many-to-many pronoun to a corresponding pronoun in an unambiguous pronoun group using a custom prompt to the LLM, wherein conversion of each identified many-to-many pronoun to the corresponding pronoun in the unambiguous pronoun group generates the plurality of secondary many-to-one pronouns.   
     
     
         15 . The one or more non-transitory machine readable information storage mediums of  claim 11 , wherein the step of creating the counterfactual context for each gender pronoun group of the plurality of predefined gender pronoun groups based on the pronoun classification context comprises:
 identifying a plurality of pronoun classifications present in the pronoun classification context using a third custom pattern finder;   mapping each identified pronoun classification with a corresponding counterfactual gender pronoun, wherein a plurality of mappings are obtained corresponding to the plurality of pronoun classifications;   appending the plurality of mappings to the one or more honorific titles;   creating a third custom pattern comprising a pronoun classification of each of the plurality of mappings, wherein each mapping is separated by the operator;   creating a lambda function that fetches a corresponding gender pronoun for each of the pronoun classifications, wherein the lambda function is utilized in a custom pattern substitution function; and   creating the counterfactual context for each gender by using the custom pattern substitution function based on the mapping of the pronoun classifications with corresponding counterfactual gender pronouns and honorific titles in the pronoun classification context.