Corpus expansion system and method thereof

Abstract

A system and method for expanding new sample seeds to automatically expand corpora, in which sample seeds are used to collect corpus is provided. The new sample seeds are generated based on the already existed sample seeds and collected corpora; The corpus expansion strategy is determined based on all the sample seeds having been used and new sample seeds: The new sample seeds are refmed based on the corpus expansion strategy, and the refmed new sample seeds are used to further collect corpus. The above steps are repeatedly executed until predefined condition is satisfied. According to the invention, corpus may be automatically expanded from the web or other resources with low cost and in convenient way to improve the coverage of corpora.

1 . A corpus expansion system, comprising: 
 a corpus collection unit, for collecting corpus using sample seeds;    a sample seed expansion unit, for generating new sample seeds, based on the sample seeds having been used by the corpus collection unit and the collected corpora;    a balancing unit, for determining a corpus expansion strategy based on all the sample seeds having been used by the corpus collection unit and the new sample seeds; and    a refining unit, for refining the new sample seeds based on the corpus expansion strategy.    
   
   
       2 . The corpus expansion system according to  claim 1 , further comprising a sample seed storing unit for storing all the sample seeds having been used by the corpus collection unit, wherein said corpus collection unit uses the refined new sample seeds to collect corpus and 
 said refining unit adds the refmed new sample seeds into the sample seed storing unit.    
   
   
       3 . The corpus expansion system according to  claim 1 , wherein, 
 said corpus expansion strategy includes at least one of the information about the class in specific field in which the sample seeds need to be expanded and the information about which sample seed need to be deleted in said new sample seeds, wherein said refining unit refines the new sample seeds based on the information.    
   
   
       4 . The corpus expansion system according to  claim 1 , wherein, 
 said sample seed expansion unit comprises at least one of:    a linguistic structure analysis based sample seed expansion unit for generating new sample seeds based on linguistic structure features and a seed annotator based sample seed expansion unit for generating new sample seeds based on known seed annotator.    
   
   
       5 . The corpus expansion system according to  claim 4 , wherein, 
 further comprising a corpus storing unit for storing the corpora collected by the corpus collection unit.    
   
   
       6 . The corpus expansion system according to  claim 4 , wherein, 
 the corpus collection unit stops collecting corpus when a predefined condition reaches.    
   
   
       7 . The corpus expansion system according to  claim 5 , wherein, 
 the predefined condition is when the number of corpora reaches a selected value, or when the ratio of the number of new sample seeds to the number of the sample seeds when expanding corpus last time is less than a specific value, or combination thereof.    
   
   
       8 . A corpus expansion method, comprising: 
 collecting corpus using sample seeds;    generating new sample seeds, based on the sample seeds having been used and the collected corpora;    determining a corpus expansion strategy based on all the sample seeds having been used and the new sample seeds;    refining the new sample seeds based on the corpus expansion strategy; and    repeating the above steps until a predefined condition reaches.    
   
   
       9 . The corpus expansion method according to  claim 8 , further including 
 storing all the sample seeds having been used when collecting corpus.    
   
   
       10 . The corpus expansion method according to  claim 8 , wherein, 
 the step of generating new sample seeds comprises at least one of generating new sample seeds based on linguistic structure features and generating new sample seeds based on known seed annotator.    
   
   
       11 . The corpus expansion method according to  claim 10 , 
 further comprising storing the collected corpora.    
   
   
       12 . The corpus expansion method according to  claim 11 , wherein, 
 the predefined condition is when the number of corpora reaches a certain value, or when the ratio of the number of new sample seeds to the number of the sample seeds when expanding corpus last time is less than a specific value.    
   
   
       13 . The corpus expansion method according to  claim 8 , wherein, 
 said corpus expansion strategy includes at least one of the information about the class in specific field in which the sample seeds need to be expanded and the information about which sample seed need to be deleted in said new sample seeds, wherein    said reforming step refines the new sample seeds based on the information.    
   
   
       14 . A program product for realizing the method as described in  claim 8 .  
   
   
       15 . A storage medium for recording the program in  claim 14.