Application Programming Interface-based Writing of Data to a Directory of a File Structure Layer of a Data Storage System

Abstract

An example method includes receiving a request to store a data item within a directory of a file structure layer for an unstructured storage layer of a data storage system; providing an identifier of a collection object representative of the directory and stored within the unstructured storage layer; sending to a plurality of storage nodes included in a cluster of the data storage system, a query as to which of the storage nodes are available to store the data item; receiving, in response to the query, replies from a subset of storage nodes that are included in the plurality of storage nodes and that are available to store the data item; selecting, based on the replies, a storage node from the subset of storage nodes; and transmitting the data item to the selected storage node for storage by the selected storage node within the unstructured storage layer.

What is claimed is: 
     
         1 . A method comprising:
 receiving, via a network, a request to store a data item within a directory of a file structure layer for an unstructured storage layer of a data storage system;   providing an identifier of a collection object representative of the directory and stored within the unstructured storage layer;   sending, via the network, to a plurality of storage nodes included in a cluster of the data storage system, a query as to which of the storage nodes are available to store the data item, wherein the nodes communicate with each other via the network to provide respective parts of the file structure layer;   receiving, via the network, in response to the query, replies from a subset of storage nodes that are included in the plurality of storage nodes and that are available to store the data item;   selecting, based on the replies, a storage node from the subset of storage nodes; and   transmitting, via the network, the data item to the selected storage node for storage by the selected storage node within the unstructured storage layer;   wherein one or more of the receiving of the request, the providing, the sending, the receiving of the replies, the selecting, or the transmitting are performed by invoking an application programming interface (API).   
     
     
         2 . The method of  claim 1 , wherein the replies from the subset of storage nodes comprise data that indicate at least one of free storage space on the subset of storage nodes, an age of hardware that the subset of storage nodes are running on, current CPU loads of the subset of storage nodes, and geographical positions of the subset of storage nodes. 
     
     
         3 . The method of  claim 2 , wherein:
 the selecting of the storage node from the subset of storage nodes comprises selecting multiple storage nodes from the subset of storage nodes; and   the transmitting comprises transmitting the data item to the multiple storage nodes for storage by the multiple storage nodes within the unstructured storage layer.   
     
     
         4 . The method of  claim 1 , wherein the data item is a data file. 
     
     
         5 . The method of  claim 1 , wherein the data item is an additional collection object representative of a subdirectory within the directory of the file structure. 
     
     
         6 . The method of  claim 1 , further comprising:
 accessing, in response to the storage of the data item by the selected storage node and based on the identifier, the collection object representative of the directory; and   updating the collection object to include a reference to the data item stored in the selected storage node.   
     
     
         7 . The method of  claim 1 , wherein the API is executed by a device configured to communicate with the data storage system by way of the network. 
     
     
         8 . The method of  claim 1 , wherein the API is executed by a storage node included in the data storage system. 
     
     
         9 . The method of  claim 1 , wherein the sending of the query comprises transmitting a multicast message to the plurality of storage nodes. 
     
     
         10 . A device comprising:
 a storage device storing instructions; and   one or more processors that, when executing the instructions, perform a process comprising:   receiving, via a network, a request to store a data item within a directory of a file structure layer for an unstructured storage layer of a data storage system;   providing an identifier of a collection object representative of the directory and stored within the unstructured storage layer;   sending, via the network, to a plurality of storage nodes included in a cluster of the data storage system, a query as to which of the storage nodes are available to store the data item, wherein the nodes communicate with each other via the network to provide respective parts of the file structure layer;   receiving, via the network, in response to the query, replies from a subset of storage nodes that are included in the plurality of storage nodes and that are available to store the data item;   selecting, based on the replies, a storage node from the subset of storage nodes; and   transmitting, via the network, the data item to the selected storage node for storage by the selected storage node within the unstructured storage layer;   wherein one or more of the receiving of the request, the providing, the sending, the receiving of the replies, the selecting, or the transmitting are performed by invoking an application programming interface (API).   
     
     
         11 . The device of  claim 10 , wherein the device is implemented by a server configured to communicate with the data storage system by way of the network. 
     
     
         12 . The device of  claim 10 , wherein the device is implemented by a storage node within the data storage system. 
     
     
         13 . The device of  claim 10 , wherein the replies from the subset of storage nodes comprise data that indicate at least one of free storage space on the subset of storage nodes, an age of hardware that the subset of storage nodes are running on, current CPU loads of the subset of storage nodes, and geographical positions of the subset of storage nodes. 
     
     
         14 . The device of  claim 13 , wherein:
 the selecting of the storage node from the subset of storage nodes comprises selecting multiple storage nodes from the subset of storage nodes; and   the transmitting comprises transmitting the data item to the multiple storage nodes for storage by the multiple storage nodes within the unstructured storage layer.   
     
     
         15 . The device of  claim 10 , wherein the data item is a data file. 
     
     
         16 . The device of  claim 10 , wherein the data item is an additional collection object representative of a subdirectory within the directory of the file structure. 
     
     
         17 . The device of  claim 10 , wherein the sending of the query comprises transmitting a multicast message to the plurality of storage nodes. 
     
     
         18 . A non-transitory computer-readable medium storing an application that, when executed, directs a device to perform a process comprising:
 receiving, via a network, a request to store a data item within a directory of a file structure layer for an unstructured storage layer of a data storage system;   providing an identifier of a collection object representative of the directory and stored within the unstructured storage layer;   sending, via the network, to a plurality of storage nodes included in a cluster of the data storage system, a query as to which of the storage nodes are available to store the data item, wherein the nodes communicate with each other via the network to provide respective parts of the file structure layer;   receiving, via the network, in response to the query, replies from a subset of storage nodes that are included in the plurality of storage nodes and that are available to store the data item;   selecting, based on the replies, a storage node from the subset of storage nodes; and   transmitting, via the network, the data item to the selected storage node for storage by the selected storage node within the unstructured storage layer;   wherein one or more of the receiving of the request, the providing, the sending, the receiving of the replies, the selecting, or the transmitting are performed by invoking an application programming interface (API).   
     
     
         19 . The non-transitory computer-readable medium of  claim 18 , wherein the device is implemented by a server configured to communicate with the data storage system by way of the network. 
     
     
         20 . The non-transitory computer-readable medium of  claim 18 , wherein the device is implemented by a storage node within the data storage system.