Method for generating answer based on advanced retrieval augmented generation and system therefor

Abstract

The disclosure relates to a high-performance RAG-based answer generation method, which includes: acquiring a query; performing a first evaluation task based on the query using a pre-trained critique model; retrieving documents related to the query based on a result of the first evaluation task; performing a second evaluation task based on the query and the retrieved documents using the critique model; and generating one or more answers, based on the query and one or more related documents, using a large language model (LLM) according to a result of the second evaluation task.

What is claimed is: 
     
         1 . A method for generating an answer based on retrieval-augmented generation (RAG) performed by a computing device, the method comprising:
 acquiring a query;   performing a first evaluation task based on the query using a pre-trained critique model;   retrieving documents related to the query based on a result of the first evaluation task;   performing a second evaluation task based on the query and the retrieved documents using the critique model; and   generating one or more answers, based on the query and one or more related documents, using a large language model (LIM) according to a result of the second evaluation task.   
     
     
         2 . The method for generating an answer of  claim 1 ,
 further comprising generating an answer corresponding to the query, based only on the query, using the LLM model according to a result of the second evaluation task.   
     
     
         3 . The method for generating an answer of  claim 1 ,
 wherein the first evaluation task is a task for determining whether to generate an answer with reference to a document retrieval result for the query or to generate an answer without referring to the document retrieval result.   
     
     
         4 . The method for generating an answer of  claim 1 ,
 wherein the critique model, when performing the first evaluation task, determines whether to refer to a document retrieval result based on the query and outputs one of a [Retrieval] token and a [No Retrieval] token based on the determination result.   
     
     
         5 . The method for generating an answer of  claim 1 ,
 further comprising resorting ranks of the retrieved documents.   
     
     
         6 . The method for generating an answer of  claim 1 ,
 wherein the second evaluation task is a task for evaluating relevance between the query and the retrieved documents.   
     
     
         7 . The method for generating an answer of  claim 1 ,
 wherein the critique model, when performing the second evaluation task, determines relevance between the query and the retrieved documents and assigns one of a [Relevant] token and an [Irrelevant] token to the retrieved documents based on the determination result.   
     
     
         8 . The method for generating an answer of  claim 1 ,
 further comprising performing a third evaluation task by inputting the one or more related documents and the one or more answers into the critique model.   
     
     
         9 . The method for generating an answer of  claim 8 ,
 wherein the third evaluation task is a task for evaluating groundedness between the related documents and the answers.   
     
     
         10 . The method for generating an answer of  claim 8 ,
 wherein the critique model, when performing the third evaluation task, determines groundedness between the related documents and the answers and assigns one of a [Fully Supported] token, a [Partially Supported] token, and a [Not Supported] token to the answers based on the determination result.   
     
     
         11 . The method for generating an answer of  claim 1 ,
 further comprising performing a fourth evaluation task by inputting the query and the one or more answers into the critique model.   
     
     
         12 . The method for generating an answer of  claim 11 ,
 wherein the fourth evaluation task is a task for evaluating a utility score between the query and the answers.   
     
     
         13 . The method for generating an answer of  claim 11 ,
 wherein the critique model, when performing the fourth evaluation task, determines a utility score between the query and the answers and assigns one of a [Utility  1 ] token to a [Utility  5 ] token to the answers based on the determination result.   
     
     
         14 . The method for generating an answer of  claim 1 , further comprising:
 calculating critique scores for the one or more answers; and   determining a final answer, based on the calculated critique scores.   
     
     
         15 . The method for generating an answer of  claim 1 ,
 wherein the critique model is generated by fine-tuning a pre-trained language model (PLM), based on learning data for respective tasks.   
     
     
         16 . The method for generating an answer of  claim 15 ,
 wherein a method of fine-tuning the PLM model is a coarse-to-fine learning method.   
     
     
         17 . The method for generating an answer of  claim 16 ,
 wherein the coarse-to-fine learning method is a method of sequentially performing zero-shot learning, one-shot learning, and few-shot learning.   
     
     
         18 . A device comprising:
 one or more processors configured to execute a plurality of operations for generating an answer based on retrieval-augmented generation (RAG); and   one or more memories configured to store a plurality of instructions for executing the plurality of operations,   wherein the plurality of operations comprise:   acquiring a query;   performing a first evaluation task based on the query using a pre-trained critique model;   retrieving documents related to the query based on a result of the first evaluation task;   performing a second evaluation task based on the query and the retrieved documents using the critique model; and   generating one or more answers, based on the query and one or more related documents, using a large language model (LIM) according to a result of the second evaluation task.   
     
     
         19 . A computer-readable storage medium storing one or more programs for generating an answer corresponding to a query by one or more processors of a computing device, the one or more programs comprising instructions for:
 acquiring a query;   performing a first evaluation task based on the query using a pre-trained critique model;   retrieving documents related to the query based on a result of the first evaluation task;   performing a second evaluation task based on the query and the retrieved documents using the critique model; and   generating one or more answers, based on the query and one or more related documents, using a large language model (LIM) according to a result of the second evaluation task.