Mechanism for managing facts in a fact repository

Abstract

Methods and systems for processing facts with one or more janitors. Facts are extracted from documents on the Internet or other sources. Facts can be any data or series of data in the documents including an attribute and a file. The data can be in the form of text, graphics, or multimedia content. Janitors transform facts responsive to inferring a certain condition associated with facts. The condition can be related to one or more of an attribute, a value, or an object of a fact being analyzed. For example, janitors can perform normalization, remove or merge similar or duplicate facts, segregate multiple values of a fact, and the like. An administrator can select which janitors are applied to facts and in which order.

1 . A computer-implemented method for processing facts extracted from web-based-documents, comprising: 
 extracting a fact from a plurality of web-based documents stored on document hosts, the fact comprising an attribute and a value, at least two of the web-based documents presenting the fact in different formats;    applying two or more janitors to the fact, each janitor inferring one or more conditions associated with the fact from at least one of the attribute and value, and responsive to inferring the one or more conditions, each of the two or more janitors transforming the fact in accordance with a different predetermined constraint for the condition by adjusting at least one of the attribute and value; and    storing the transformed fact in a fact repository.    
   
   
       2 . The method of  claim 1 , wherein the applying two or more janitors comprises: 
 applying a first and a second janitor in a predetermined order to infer a first condition and a second condition, respectively, wherein the second janitor is able to detect the second condition responsive to applying the first janitor.    
   
   
       3 . A computer-implemented method for processing facts extracted from web-based-documents, comprising: 
 extracting a fact from an unstructured web-based document, the fact containing an attribute and a value;    inferring a condition associated with the fact from at least one of the attribute and value;    responsive to inferring the condition, transforming the fact to a predetermined constraint for the condition by adjusting at least one of the attribute and value; and    storing the transformed fact in a fact repository.    
   
   
       4 . The method of  claim 3 , wherein: 
 inferring the condition comprises inferring a type for the fact, and    transforming the fact comprises transforming the fact to a predetermined constraint for the fact type.    
   
   
       5 . The method of  claim 3 , further comprising: 
 receiving another fact,    wherein inferring the condition comprises inferring that the facts have the same condition, and    wherein transforming the fact comprises comparing the facts.    
   
   
       6 . The method of  claim 3 , further comprising: 
 receiving another fact containing another attribute and another value,    wherein inferring the condition comprises detecting that at least one of the attributes or the values are similar,    wherein transforming the fact comprises merging the facts.    
   
   
       7 . The method of  claim 3 , wherein: 
 transforming the fact comprises reformatting the value according to the predetermined constraint.    
   
   
       8 . The method of  claim 3 , wherein: 
 inferring the condition comprises inferring that the value includes two or more independent values, and    transforming the fact comprises generating two or more new facts, each one of the two or more new facts having one of the two or more independent values.    
   
   
       9 . The method of  claim 3 , wherein: 
 inferring the condition comprises inferring the condition based on an object associated with the fact, the object being common to both the fact and at least one other fact.    
   
   
       10 . The method of  claim 3 , further comprising: 
 retrieving the fact from storage in the fact repository.    
   
   
       11 . The method of  claim 3 , further comprising: 
 receiving the fact during extraction from the web-based document.    
   
   
       12 . The method of  claim 3 , wherein the web-based document is encoded in Hypertext Markup Language, and the fact is extracted from an HTML table in the document.  
   
   
       13 . A computer-implemented method for processing facts extracted from documents, comprising: 
 extracting a fact from an unstructured web-based document, the fact containing an attribute and a value;    applying a collection of janitors in a predetermined order to process the fact, each janitor configured to infer a different condition associated with a fact from at least one of the attribute and value, and responsive to the janitor inferring the specific condition, the janitor transforming the fact to a different predetermined constraint for the condition by adjusting at least one of the attribute and value, and responsive to the janitor failing to infer the specific condition, the janitor discontinuing processing of the fact; and    storing the fact in a fact repository of the computer.    
   
   
       14 . A computer program product stored on a computer readable medium and configured to perform a method for processing facts extracted from unstructured web-based documents and stored in a repository, comprising: 
 extracting a fact containing an attribute and a value, the fact extracted from an unstructured web-based document;    inferring a condition associated with the fact from at least one of the attribute and value;    responsive to inferring the condition, transforming the fact to a predetermined constraint for the condition by adjusting at least one of the attribute and value; and    storing the facts in the fact repository.    
   
   
       15 . The computer program product of  claim 14 , wherein: 
 inferring the condition comprises inferring a type for the fact, and    transforming the fact comprises transforming the fact to a predetermined constraint for the fact type.    
   
   
       16 . The computer program product of  claim 14 , further comprising: 
 receiving another fact,    wherein inferring the condition comprises inferring that the facts have the same condition, and    wherein transforming the fact comprises normalizing the facts.    
   
   
       17 . The computer program product of  claim 14 , further comprising: 
 receiving another fact containing another attribute and another value,    wherein inferring the condition comprises detecting that at least one of the attributes or the values are similar,    wherein transforming the fact comprises merging the facts.    
   
   
       18 . The computer program product of  claim 14 , wherein: 
 transforming the fact comprises reformatting the value according to the predetermined constraint.    
   
   
       19 . The computer program product of  claim 14 , wherein: 
 inferring the condition comprises inferring that the value includes two or more independent values, and    transforming the fact comprises generating two or more new facts, each one of the two or more new facts having one of the two or more independent values.    
   
   
       20 . The computer program product of  claim 14 , wherein: 
 inferring the condition comprises inferring the condition based on an object associated with the fact, the object used as a common indexer for both the fact and at least one other fact.    
   
   
       21 . The computer program product of  claim 14 , further comprising: 
 retrieving the fact from storage in the fact repository.    
   
   
       22 . The computer program product of  claim 14 , further comprising: 
 receiving the fact during extraction from the document.    
   
   
       23 . The computer program product of  claim 14 , wherein the web-based document is encoded in Hypertext Markup Language, and the fact is extracted from a table as indicated within the encoding.  
   
   
       24 . A system for processing facts extracted from documents, comprising: 
 an extractor to extract facts, the facts each containing an attribute and a value, the facts extracted from a plurality of unstructured web-based documents;    two or more janitors configured to receive a fact, each of the two or more janitors configured to operate in a predetermined order and to infer a different specific condition associated with the fact from at least one of the attribute and value, the two or more janitors configured to transform the fact to a predetermined constraint for the condition by adjusting at least one of the attribute and value, and store the fact.    
   
   
       25 . The system of  claim 24 , further comprising: 
 a script module, in communication with the one or more janitors, the script to describe an order for the one or more janitors to process the fact.    
   
   
       26 . A method performed by a software janitor in a data processing system, comprising: 
 receiving a first fact containing an attribute and a value, the value having a first format when it was been extracted from an unstructured web-based document;    receiving a plurality of existing facts, the plurality of facts each containing an attribute and a value, each value having been previously extracted from an unstructured web-based document, and at least one of the values having had a second format different from the first format at the time the existing fact was extracted from the web-based document; and    comparing the value of the first fact to values of the existing facts to determine whether the first fact should be stored in a fact repository of the data processing system.    
   
   
       27 . The method of  claim 26 , wherein comparing to determine whether the first fact should be stored in the fact repository comprises: not storing the fact in the fact repository if it duplicates a threshold number of the existing facts.  
   
   
       28 . The method of  claim 26 , wherein comparing to determine whether the first fact should be stored in the fact repository comprises storing the fact in the fact repository if it is corroborated by a threshold number of the existing facts.  
   
   
       29 . The method of  claim 26 , wherein comparing to determine whether the first fact should be stored in the fact repository comprises refraining from storing the fact in the fact repository if it is not corroborated by a threshold number of the existing facts.  
   
   
       30 . The method of  claim 26 , further comprising: normalizing the first fact and the at least one existing fact prior to comparing the first fact and the existing facts.  
   
   
       31 . The method of  claim 26 , wherein comparing the first fact and the existing facts further comprises being able to compare facts in the first and second formats.  
   
   
       32 . The method of  claim 26 , wherein receiving the plurality of existing facts comprises receiving the plurality of existing facts from the fact repository.