Storing data across a plurality of storage nodes

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for storing data on storage nodes. In one aspect, a method includes receiving a file to be stored across a plurality of storage nodes each including a cache. The is stored by storing portions of the file each on a different storage node. A first portion is written to a first storage node's cache until determining that the first storage node's cache is full. A different second storage node is selected in response to determining that the first storage node's cache is full. For each portion of the file, a location of the portion is recorded, the location indicating at least a storage node storing the portion.

What is claimed is: 
     
         1 - 29 . (canceled) 
     
     
         30 . A computer-implemented method comprising:
 receiving a file to be stored across a plurality of storage nodes;   storing one or more portions of the file on one or more of the plurality of storage nodes;   identifying a data access pattern associated with the file;   determining a quantity of one or more additional storage nodes of the plurality of storage nodes for storage of the file based on the identified data access pattern; and   storing additional portions of the file on the one more additional storage nodes.   
     
     
         31 . The method of  claim 30 , wherein identifying the data access pattern associated with the file includes monitoring a current data access pattern of stored portions of the file. 
     
     
         32 . The method of  claim 30 , wherein identifying the data access pattern associated with the file includes identifying a projected data access pattern of the file. 
     
     
         33 . The method of  claim 30 , wherein identifying the data access pattern associated with the file includes identifying a previous data access pattern of the file prior to the current storing of the file. 
     
     
         34 . The method of  claim 30 , wherein determining the quantify of the one or more additional storage nodes of the plurality of storage nodes for storage of the file is further based on a timestamp associated with one or more portions of the file. 
     
     
         35 . The method of  claim 30 , further comprising classifying the file as high-demand based on the identified data access pattern, and in response, storing the additional portion of the file on at least two or more of the additional storage nodes. 
     
     
         36 . The method of  claim 30 , wherein storing the additional portions of the file on the one or more additional storage nodes further comprises minimizing storing a threshold quantity of portions of the file on two or fewer of the additional storage nodes. 
     
     
         37 . A system comprising:
 one or more computers; and   a computer-readable medium coupled to the one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising:
 receiving a file to be stored across a plurality of storage nodes; 
 storing one or more portions of the file on one or more of the plurality of storage nodes; 
 identifying a data access pattern associated with the file; 
 determining a quantity of one or more additional storage nodes of the plurality of storage nodes for storage of the file based on the identified data access pattern; and 
 storing additional portions of the file on the one more additional storage nodes. 
   
     
     
         38 . The system of  claim 37 , wherein identifying the data access pattern associated with the file includes monitoring a current data access pattern of stored portions of the file. 
     
     
         39 . The system of  claim 37 , wherein identifying the data access pattern associated with the file includes identifying a projected data access pattern of the file. 
     
     
         40 . The system of  claim 37 , wherein identifying the data access pattern associated with the file includes identifying a previous data access pattern of the file prior to the current storing of the file. 
     
     
         41 . The system of  claim 37 , wherein determining the quantify of the one or more additional storage nodes of the plurality of storage nodes for storage of the file is further based on a timestamp associated with one or more portions of the file. 
     
     
         42 . The system of  claim 37 , the operations further comprising classifying the file as high-demand based on the identified data access pattern, and in response, storing the additional portion of the file on at least two or more of the additional storage nodes. 
     
     
         43 . The system of  claim 37 , wherein storing the additional portions of the file on the one or more additional storage nodes further comprises minimizing storing a threshold quantity of portions of the file on two or fewer of the additional storage nodes. 
     
     
         44 . A computer storage medium encoded with a computer program, the program comprising instructions that when executed by data processing apparatus cause the data processing apparatus to perform operations comprising:
 receiving a file to be stored across a plurality of storage nodes;   storing one or more portions of the file on one or more of the plurality of storage nodes;   identifying a data access pattern associated with the file;   determining a quantity of one or more additional storage nodes of the plurality of storage nodes for storage of the file based on the identified data access pattern; and   storing additional portions of the file on the one more additional storage nodes.   
     
     
         45 . The computer storage medium of  claim 44 , wherein identifying the data access pattern associated with the file includes monitoring a current data access pattern of stored portions of the file. 
     
     
         46 . The computer storage medium of  claim 44 , wherein identifying the data access pattern associated with the file includes identifying a projected data access pattern of the file. 
     
     
         47 . The computer storage medium of  claim 44 , wherein identifying the data access pattern associated with the file includes identifying a previous data access pattern of the file prior to the current storing of the file. 
     
     
         48 . The computer storage medium of  claim 44 , wherein determining the quantify of the one or more additional storage nodes of the plurality of storage nodes for storage of the file is further based on a timestamp associated with one or more portions of the file. 
     
     
         49 . The computer storage medium of  claim 44 , the operations further comprising classifying the file as high-demand based on the identified data access pattern, and in response, storing the additional portion of the file on at least two or more of the additional storage nodes.