Optimal storage and retrieval of xml data

Abstract

A document management system manages a large number of XML documents on an efficient and cost-effective basis. Storage requirements are reduced, because compressed versions of the XML documents, which are much smaller in size than the XML documents themselves, are used when processing queries. Processing requirements are reduced, because parsing is not a required step when processing queries. Instead of parsing, the query is processed by unpacking the compressed version of the document identified in the query, node by node until enough information has been decoded to satisfy the query. Processing speed is improved in two ways. First, unpacking as carried out according to the invention is a much faster process than parsing. Second, the entire document need not be unpacked.

1 : A document management system managing documents having a hierarchical structure for responding to a query, the document management system comprising:
 a first storage section containing binary objects corresponding to documents having a hierarchical structure;   a second storage section including at least two dictionaries of terms, each dictionary associated with an identified type of node including corresponding terms, each term having a unique value associated therewith in the dictionary; and   a processor for generating a query result using the binary objects and at least one of the at least two dictionaries in response to queries.   
   
   
       2 : The document management system according to  claim 1 , further comprising a master storage section comprising the documents. 
   
   
       3 : The document management system according to  claim 2 , wherein the processor is operable for accessing the first storage section during processing of a query but not the master storage section. 
   
   
       4 : The document management system according to  claim 1 , wherein the documents are XML documents and the binary objects are derived from the XML documents that have been parsed. 
   
   
       5 : The document management system according to  claim 4 , wherein the at least two dictionaries comprise a first dictionary, the first dictionary of terms comprises terms corresponding to a node of a first type and a second dictionary of terms comprises terms corresponding to a node of a second type. 
   
   
       6 : The document management system according to  claim 1 , wherein the unique value associated with each term in the dictionary corresponds to a positional offset in memory of said each term with respect to the beginning of the dictionary. 
   
   
       7 : A method of managing documents having a hierarchical structure for responding to a query, the method comprising:
 receiving a query; and   generating a query result using binary objects and at least one of at least two dictionaries of terms, wherein each dictionary of the at least two dictionaries is associated with an identified type of node including corresponding terms in the documents, each term having a unique value associated therewith in the dictionary, and the documents are stored as binary objects using the unique values in place of the terms.   
   
   
       8 : The method of  claim 7 , wherein the binary objects are stored in a database with keys that identify the documents. 
   
   
       9 : The method of  claim 8 , further comprising:
 accessing a database to retrieve the binary object associated with a document identified in the query; and   comparing the query against the binary object for generating the query result.   
   
   
       10 : The method according to  claim 9 , wherein comparing includes conducting first search through nodes of the document that are represented in the binary object. 
   
   
       11 : The method according to  claim 9 , wherein comparing comprises:
 retrieving a term associated with a root node of the document represented in the binary object; and   comparing a root node identified in the query with the retrieved term.   
   
   
       12 : The method according to  claim 11 , wherein, if the root node identified in the query matches the retrieved term and there are additional node levels in the query, comparing further comprises:
 retrieving terms associated with child nodes of the document that are represented in the binary object; and   comparing the next level node identified in the query with the retrieved terms.   
   
   
       13 : The method according to  claim 12 , wherein, if there is a match between the next level node and one of the retrieved terms, the query is processed with respect to those nodes that originate from the child node having a term that matches the next level node, and not with respect to those nodes that originate from the other child nodes. 
   
   
       14 : The method according to  claim 11 , wherein the term is retrieved based on a value associated with the term from the dictionary that associates each of a plurality of different values, including said value, with a unique term. 
   
   
       15 : The method according to  claim 9 , wherein the documents are XML documents and the query comprises an XPATH query.