Method and apparatus for learnning language model from stylistic point of view, and recording medium for recording the same

Abstract

Disclosed is a method of training a language model from a stylistic perspective, and the method includes: a first step of pre-training a language model using an unsupervised training method using a first training dataset; a second step of re-training the pre-trained language model using a second training dataset with distinguished styles; and a third step of fine-tuning the re-trained language model using a third training dataset classified by domain through supervised learning.

What is claimed is: 
     
         1 . A method of training a language model from a stylistic perspective, the method comprising:
 a first step of pre-training a language model using an unsupervised training method using a first training dataset;   a second step of re-training the pre-trained language model using a second training dataset with distinguished styles; and   a third step of fine-tuning the re-trained language model using a third training dataset classified by domain through supervised learning.   
     
     
         2 . The method of  claim 1 , wherein the first training dataset is a large corpus created regardless of the domain and the styles. 
     
     
         3 . The method of  claim 1 , wherein in the second step, the pre-trained language model is re-trained through unsupervised learning. 
     
     
         4 . The method of  claim 1 , wherein the second training dataset has a domain identical to a domain of the third training dataset. 
     
     
         5 . The method of  claim 1 , wherein the second training dataset is a dataset different from the first training dataset and has a domain identical to a domain of the third training dataset. 
     
     
         6 . A non-transitory computer-readable recording medium in which a program for causing a computer to execute the method according to  claim 1 . 
     
     
         7 . A computing device comprising:
 a memory configured to store a language model for machine reading comprehension; and   a processor configured to execute the language model and infers a result from an input,   wherein the language model is pre-trained using a first training dataset through unsupervised learning, re-trained using a second training dataset with distinguished styles through unsupervised learning, and fine-tuned using a third training dataset classified by domain through supervised learning.   
     
     
         8 . The computing device of  claim 7 , wherein the first training dataset is a large corpus created regardless of the domain and the styles. 
     
     
         9 . The computing device of  claim 7 , wherein the second training dataset has a domain identical to a domain of the third training dataset. 
     
     
         10 . The computing device of  claim 7 , wherein the second training dataset is a dataset different from the first training dataset and has a domain identical to a domain of the third training dataset.