Zero shot detection of llm generated phishing emails

Abstract

A pipeline for classifying malicious communications as AI generated or human generated has been created. The pipeline uses a first prompt template that directs a first LLM to parse a phishing e-mail and extract information from the phishing e-mail. The pipeline searches publicly available information to obtain current information based on keywords in the information extracted from the phishing e-mail. The pipeline then uses a second LLM to compose an e-mail. With a different prompt template, the pipeline directs the second LLM to compose an e-mail based on the obtained, current information and a recipient and sender extracted from the phishing e-mail. With another prompt, the pipeline directs the second LLM to determine whether the phishing e-mail is similar to the LLM composed e-mail. If the second LLM responds that the phishing e-mail is similar to the composed e-mail, then the phishing e-mail is classified as AI generated.

1 . A method comprising:
 prompting a first language model to extract from a first electronic communication a header and keywords from a body of the first electronic communication, wherein the first electronic communication has already been determined to be an attack;   searching publicly available information based on the keywords;   prompting a second language model to compose an electronic communication based, at least in part, on information acquired from the searching and a sender and a recipient indicated in the header;   prompting the second language model to determine whether the first and second electronic communications are similar; and   indicating the first electronic communication as generated by artificial intelligence if the second language model responds that the first and second electronic communications are similar.   
     
     
         2 . The method of  claim 1 , wherein prompting the second language model to determine whether the first and second electronic communications are similar is according to zero shot prompting. 
     
     
         3 . The method of  claim 1  further comprising removing personally identifiable information from the first and second electronic communications before prompting the second language model to determine whether the first and second electronic communications are similar. 
     
     
         4 . The method of  claim 1 , wherein searching publicly available information comprises prompting the first language model or a third language model to search publicly available information based on the keywords. 
     
     
         5 . The method of  claim 1 , wherein prompting the first language model comprises:
 generating a first prompt with one or more task instructions to extract a sender and a recipient from the first electronic communication, to extract the body from the first electronic communication, to remove indication of the sender and the recipient from the extracted body and identify keywords in the extracted body after removal of the sender and the recipient; and   submitting the first prompt to the first language model.   
     
     
         6 . The method of  claim 1 , wherein prompting the second language model to determine whether the first and second electronic communications are similar comprises:
 generating a first prompt with a set of one or more task instructions to determine similarity based on topic of content in the bodies of the first and second electronic communications and disregard recipient and sender; and   submit the first prompt to the second language model.   
     
     
         7 . The method of  claim 6 , wherein generating the first prompt with the set of one or more task instructions to determine similarity comprises generate the first prompt with the set of one or more task instructions to also disregard style and parts of the electronic communications that are not the bodies. 
     
     
         8 . A non-transitory, machine-readable medium having program code stored thereon, the program code comprising instructions to:
 classify a malicious communication as artificial intelligence (AI) generated or not AI generated, wherein the instructions to classify the malicious communication comprise instructions to,
 prompt a first language model to extract keywords from a body of the malicious communication; 
 retrieve current publicly available information based on the keywords; 
 prompt a second language model to compose a communication based, at least in part, on the retrieved information and a sender and a recipient indicated in the malicious communication; 
 prompt the second language model to determine whether the malicious communication is similar to the composed communication based, at least in part, on content of the communications; and 
 wherein the instructions to classify the malicious communication comprise the instructions to classify the malicious communication as AI generated if the second language model responds that the malicious communication is similar to the composed communication. 
   
     
     
         9 . The non-transitory, machine-readable medium of  claim 8 , wherein the instructions to prompt the second language model to determine whether the malicious communication is similar to the composed communication comprise instructions to generate a zero-shot prompt for the second language model to determine similarity. 
     
     
         10 . The non-transitory, machine-readable medium of  claim 8 , wherein the program code further comprises instructions to remove personally identifiable information from the communications before the similarity determination. 
     
     
         11 . The non-transitory, machine-readable medium of  claim 8 , wherein the instructions to prompt the second language model to determine whether the malicious communication is similar to the composed communication comprise instructions to generate a prompt with a set of one or more task instructions to remove personally identifiable information from the communications and then determine whether the malicious communication is similar to the composed communication. 
     
     
         12 . The non-transitory, machine-readable medium of  claim 8 , wherein the instructions to retrieve current public available information based on the keywords comprise instructions to invoke a crawler or instruct a model to search publicly available information for current information based on the keywords. 
     
     
         13 . The non-transitory, machine-readable medium of  claim 8 , wherein the instructions to prompt the first language model comprise instructions to:
 generate a first prompt with one or more task instructions to extract a sender and a recipient from the malicious communication, to extract content from the body of the malicious communication, to remove indication of the sender and the recipient from the extracted content and identify keywords in the extracted content after removal of the sender and the recipient; and   submit the first prompt to the first language model.   
     
     
         14 . The non-transitory, machine-readable medium of  claim 8 , wherein the instructions to prompt the second language model to determine whether the malicious communication is similar to the composed communication comprise instructions to:
 generate a first prompt with a set of one or more task instructions to determine similarity based on topic of the content of the communications and disregard recipient and sender; and   submit the first prompt to the second language model.   
     
     
         15 . The non-transitory, machine-readable medium of  claim 14 , wherein the instructions to generate the first prompt with the set of one or more task instructions to determine similarity comprise the instructions to generate the first prompt with the set of one or more task instructions to also disregard style and parts of the communications that are not the bodies. 
     
     
         16 . An apparatus comprising:
 a processor;   a machine-readable medium having instructions stored thereon, the instructions executable by the processor to cause the apparatus to:   classify a malicious communication as artificial intelligence (AI) generated or not AI generated, wherein the instructions to classify the malicious communication comprise instructions to,
 prompt a first language model to extract keywords from a body of the malicious communication; 
 retrieve current publicly available information based on the keywords; 
 prompt a second language model to compose a communication based, at least in part, on the retrieved information and a sender and a recipient indicated in the malicious communication; 
 prompt the second language model to determine whether the malicious communication is similar to the composed communication based, at least in part, on content of the communications; and 
 wherein the instructions to classify the malicious communication comprise the instructions to classify the malicious communication as AI generated if the second language model responds that the malicious communication is similar to the composed communication. 
   
     
     
         17 . The apparatus of  claim 16 , wherein the machine-readable medium further has stored thereon instructions executable by the processor to cause the apparatus to remove personally identifiable information from the communications before the similarity determination. 
     
     
         18 . The apparatus of  claim 16 , wherein the instructions to prompt the second language model to determine whether the malicious communication is similar to the composed communication comprise instructions executable by the processor to cause the apparatus to generate a prompt with a set of one or more task instructions to remove personally identifiable information from the communications and then determine whether the malicious communication is similar to the composed communication. 
     
     
         19 . The apparatus of  claim 16 , wherein the instructions to prompt the first language model comprise instructions executable by the processor to cause the apparatus to:
 generate a first prompt with one or more task instructions to extract a sender and a recipient from the malicious communication, to extract content from the body of the malicious communication, to remove indication of the sender and the recipient from the extracted content and identify keywords in the extracted content after removal of the sender and the recipient; and   submit the first prompt to the first language model.   
     
     
         20 . The apparatus of  claim 16 , wherein the instructions to prompt the second language model to determine whether the malicious communication is similar to the composed communication comprise instructions executable by the processor to cause the apparatus to:
 generate a first prompt with a set of one or more task instructions to determine similarity based on topic of the content of the communications and disregard recipient and sender; and   submit the first prompt to the second language model.