Data archiving system

Abstract

An encrypted file storage solution consists of a cluster of processing nodes, external data storage, and a software agent (the “File System Watcher”), which is installed on the application servers. Cluster sizes of one node up to many hundreds of nodes are possible. There are also remote “Key Servers” which provide various services to one or more clusters. The preceding describes a preferred embodiment, though in some cases it may be desirable to “collapse” some of the functionality into a smaller number of hardware devices, typically trading off cost versus security and fault-tolerance.

1 - 8 . (canceled) 
     
     
         9 . A computer-implemented method of archiving data, comprising:
 receiving at a data archiving computer system data files for storage from a client node;   encrypting with the data archiving computer system each data file received for storage with a respective unique encryption key;   storing each encrypted data file in asset storage of the data archiving computer system;   generating with the data archiving computer system a list of one or more of the data files marked for deletion from the asset storage; and   deleting the one or more data files in the list from the asset storage by deleting the respective unique encryption keys of each data file in the list.   
     
     
         10 . The computer-implemented method of  claim 9 , further comprising:
 requesting code pages from a key server;   generating and storing a database of the respective unique encryption keys in one or more key containers contained in the requested code pages; and   encrypting the one or more key containers with a master key.   
     
     
         11 . The computer-implemented method of  claim 10 , further comprising retiring the master key after a predetermined period of time, wherein retiring the master key includes:
 obtaining a new master key;   decrypting the one or more key containers with the master key being retired;   encrypting the one or more key containers with the new master key; and   locking away the master key being retired, wherein the master key being retired is destroyed.   
     
     
         12 . The computer-implemented method of  claim 9 , further comprising the data archiving computer system performing steps of:
 creating an associative cryptographic hash of each data file received;   creating an associative cryptographic hash of each encrypted data file;   storing each encrypted data file and its associative cryptographic hash to the asset storage;   re-computing the associative cryptographic hash of a given one of the encrypted data files;   verifying authenticity of the given encrypted file by comparing the associative cryptographic hash to the re-computed cryptographic hash; and   reporting an error when comparing the associative cryptographic hash to the re-computed cryptographic hash indicates a mismatch.   
     
     
         13 . The computer-implemented method of  claim 9 , further comprising the data archiving computer system performing steps of:
 creating an associative cryptographic hash of each data file received;   storing the associative cryptographic hash of each data file received to the asset storage;   creating an associative cryptographic hash for a file on the client node;   comparing the associative cryptographic hash of the file on the client node to the list of one or more files marked for deletion from the asset storage; and   notifying the client node of existence of an uncontrolled copy of one or more of the data files marked for deletion.   
     
     
         14 - 19 . (canceled) 
     
     
         20 . A computer-implemented method of deleting files, comprising:
 generating with a data archiving computer system a list of one or more files marked for deletion from asset storage, each of the one or more files being encrypted with a respective unique encryption key and stored as an asset in the asset storage; and   storing the respective unique encryption keys in a key container, the key container contained in a cached code page;   deleting the one or more files in the list from the asset storage by deleting the respective unique encryption keys from the key container.   
     
     
         21 . The computer-implemented method of  claim 20 , wherein the respective unique encryption keys are required to decrypt the files in the list, such that deletion of the files in the list is effectively achieved by deletion of the respective unique encryption keys. 
     
     
         22 . The computer-implemented method of  claim 20 , further comprising obtaining the cached code page from a key server. 
     
     
         23 . The computer-implemented method of  claim 22 , wherein the code page is locally cached and the key server is remote. 
     
     
         24 . The computer-implemented method of  claim 22 , further comprising requesting another code page from the key server when a number of cached encryption keys falls below a threshold. 
     
     
         25 . The computer-implemented method of  claim 20 , further comprising:
 receiving an updated code page from the key server; and   overwriting the code page with the updated code page supplied by the key server, wherein expired encryption keys are deleted.   
     
     
         26 . The computer-implemented method of  claim 25 , further comprising the data archiving computer system instructing the key server to remove the respective unique encryption keys associated with the deleted files in the list, wherein the updated code page no longer contains the removed encryption keys. 
     
     
         27 . The computer-implemented method of  claim 25 , wherein ability to decrypt expired assets associated with the expired encryption keys is lost and the expired assets persisting on backup media are effectively irrecoverable. 
     
     
         28 . A data archiving system, comprising:
 an application server configured to receive data files for storage from a client node, the application server comprising asset storage and computer processor nodes configured to:
 encrypt the data files for storage as assets in the asset storage, wherein each of the data files is encrypted with a respective unique encryption key; 
 generate a list of one or more of the data files for deletion from the asset storage, wherein the respective unique encryption keys are required to decrypt the files in the list; and 
 delete the one or more files from the asset storage by deleting the respective unique encryption key of each data file on the list. 
   
     
     
         29 . The data archiving system of  claim 28 , further comprising a local cache of code pages, the code pages containing key containers storing the respective unique encryption keys. 
     
     
         30 . The data archiving system of  claim 28 , further comprising a key manager process configured to obtain the code pages from a key server. 
     
     
         31 . The data archiving system of  claim 30 , wherein the key manager process is configured to request a set of the code pages from the key server. 
     
     
         32 . The data archiving system of  claim 30 , wherein the key manager process is configured to overwrite the code pages with updated code pages supplied by the key server, wherein expired encryption keys are deleted. 
     
     
         33 . The data archiving system of  claim 30 , wherein the key manager process is configured to instruct the key server to remove the respective unique encryption keys associated with the deleted files in the list. 
     
     
         34 . The data archiving system of  claim 25 , further comprising a disposition manager process configured to report a list of the respective unique encryption keys to be deleted, such that expired assets associated with the expired encryption keys and persisting on backup media are effectively irrecoverable.