Distributed storage and communication

Abstract

Storing, retrieving, transmitting and receiving data ( 20 ) by a) separating the data into a plurality of data subsets (A, B); b) generating parity data (P) from the plurality of data subsets (A, B) such that any one or more of the plurality of data subsets may be recreated from the remaining data subsets and the parity data (P). Steps a and b may be repeated on one or more of the plurality of data subsets and parity data providing further data subsets and further parity data; and d) storing each of the further data subsets and further parity data in separate storage locations ( 380 ) or transmitting the further data subsets and further parity data.

1 - 65 . (canceled) 
     
     
         66 . A method of retrieving a data set, the method comprising:
 recovering subsets of data and parity data from separate storage locations that are remote from each other and accessed over an external network;   recreating any missing subsets of data from the recovered subsets of data and recovered parity data to form recreated subsets of data, the recovered subsets of data, recovered parity data, and recreated subsets of data providing a second level cluster;   combining the recovered subsets of data and recreated subsets of data of the second level cluster to form a plurality of recombined subsets of data and recombined parity data to provide a first level cluster;   recreating any subsets of data missing from the first level cluster from the recombined subsets of data and the recombined parity data to form recreated subsets of data, the recreated subsets of data forming part of the first level cluster; and   combining the recombined subsets of data and recreated subsets of data of the first level cluster to form the data set.   
     
     
         67 . The method of  claim 66 , wherein the separate physical locations are each located at separate physical devices. 
     
     
         68 . The method of  claim 66 , wherein the original data is encrypted, the method further comprising decrypting the original data. 
     
     
         69 . The method of  claim 66 , further comprising receiving location information of the separate storage locations. 
     
     
         70 . The method of  claim 66 , wherein the separate storage locations comprise at least one of a hard disk drive, an optical disk, a FLASH RAM, a web server, an FTP server, or a network file server. 
     
     
         71 . The method of  claim 66 , further comprising:
 recovering authentication codes associated with one or more of the subsets of data and parity data;   authenticating one or more of the subsets of data and parity data using the associated authentication codes; and   recreating any subsets of data that fail authentication from the recovered subsets of data and parity data to form recreated subsets of data within the second level cluster.   
     
     
         72 . The method of  claim 71 , wherein the authentication codes are hash codes and the authenticating comprises:
 applying a hash function to the subsets of data and/or parity data to generate a comparison hash code; and   comparing the comparison hash code with the authentication codes associated with the data subsets and/or parity data.   
     
     
         73 . An apparatus for retrieving a data set, the apparatus comprising a processor arranged to:
 recover subsets of data and parity data from separate storage locations that are remote from each other and accessed over an external network;   recreate any missing subsets of data from the recovered subsets of data and recovered parity data to form recreated subsets of data, the recovered subsets of data, recovered parity data, and recreated subsets of data providing a second level cluster;   combine the recovered subsets of data and recreated subsets of data of the second level cluster to form a plurality of recombined subsets of data and recombined parity data to provide a first level cluster;   recreate any subsets of data missing from the first level cluster from the recombined subsets of data and the recombined parity data to form recreated subsets of data, the recreated subsets of data forming part of the first level cluster; and   combine the recombined subsets of data and any recreated subsets of data of the first level cluster to form the data set.   
     
     
         74 . The apparatus of  claim 73 , wherein the processor is further arranged to:
 recover authentication codes associated with the one or more of the recovered subsets of data and recovered parity data of the second level cluster;   authenticate one or more of the recovered subsets of data and recovered parity data using the associated authentication codes; and   recreate any recovered subsets of data that fail authentication from the recovered subsets of data and recovered parity data to form recreated subsets of data within the second level cluster.   
     
     
         75 . The apparatus of  claim 73 , further comprising storage locations. 
     
     
         76 . A data storage medium for storing a data file, the data storage medium comprising data subsets, parity data, and authentication codes, wherein:
 the data subsets are combinable to produce further data subsets;   the further data subsets are combinable to produce the data file;   the authentication codes provide authentication of the data subsets; and   the parity data are combinable with the data subsets to regenerate missing data subsets or data subsets which fail authentication.   
     
     
         77 . The data storage medium of  claim 76 , wherein the data storage medium comprises at least one of a compact disc, a DVD, a hard drive, a solid state drive, a FLASH memory, or a digital tape. 
     
     
         78 . The data storage medium of  claim 76 , wherein the data file comprises at least one of multimedia data, audio data, video data, MPEG data, MP3 data, music data, database data, or binary data.