Large scale storage system

Abstract

A distributed storage system comprising: an infrastructure layer including interconnected computer nodes, wherein: each one of the interconnected computer nodes comprising processing resources configured to execute a Unified Distributed Storage Platform (UDSP) agent configured to: receive a task having assignments, calculating grades for assignments such that each grade is indicative of a suitability to execute assignments while meeting at least one Service Level Specification (SLS) requirement. The grade is calculated based on storage related resources parameters data. The task is routed to a more suitable computer node based on the calculated grades. The updated infrastructure layer is created in response to adding interconnected computer nodes and executing assignments of following tasks or routing the following tasks to a more suitable computer node based on the calculated grades.

1 . A computer node configured to being connected to an infrastructure layer of a distributed storage system, said infrastructure layer including interconnected computer nodes, the computer node comprising:
 at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent, wherein said UDSP agent is configured to:   receive at least one Service Level Specification (SLS) comprising user-defined storage requirements referring to at least one logical storage entity, and storage-related resources parameters data relating to one or more storage-related resources connected to said interconnected computer nodes;   calculate a configuration for the distributed storage system based, at least, on said at least one SLS and said storage-related resources parameters data;   automatically allocate at least part of one of said storage-related resources according to the calculated configuration;   receive dynamic behavior parameters data relating to dynamic behavior of the distributed storage system;   upon at least one SLS being breached, calculate a reconfiguration for the distributed storage system, based, at least, on said at least one SLS, said storage-related resources parameters data and said dynamic behavior parameters data; and   automatically allocate at least part of one of said storage-related resources according to the calculated reconfiguration.   
     
     
         2 . The computer node of  claim 1 , wherein said UDSP agent is further configured to:
 receive a task comprising at least one assignment;
 calculate grades for at least one of said assignments in respect of at least one of said interconnected computer nodes, such that each grade being indicative of a suitability of a respective computer node of said interconnected computer nodes to execute a respective assignment of said assignments while meeting at least one user-defined storage-related requirement of said at least one SLS, said grade is calculated also based on parameters data relating to one or more storage-related resources connected to said respective computer node, if any; 
 execute one or more of said assignments or route the task to a more suitable computer node, based on the calculated grades; 
 calculate grades for assignments of following tasks in respect of at least one added interconnected computer node of an updated infrastructure layer; said updated infrastructure layer is created in response to adding at least one additional interconnected computer node thereto; and 
 execute one or more of said assignments of following tasks or route said following tasks to a more suitable computer node of said updated infrastructure layer, based on the calculated grades. 
   
     
     
         3 . The computer node of  claim 1 , wherein said configuration is calculated by an optimization engine. 
     
     
         4 . The computer node of  claim 1 , wherein said at least one user-defined storage-related requirement includes at least one of:
 a. location;   b. local protection level;   c. backup retention policy;   d. remote protection level;   e. performance levels   f. encryption level;   g. de-duplication;   h. compression;   i. storage method.   
     
     
         5 . The computer node of  claim 1 , wherein said one or more storage-related resources include at least one storage-related resource of the following storage-related resource categories: cache resources, data storage resources and network resources. 
     
     
         6 . The computer node of  claim 1 , wherein said storage-related resources are distributed among at least two locations. 
     
     
         7 . The computer node of  claim 1 , wherein said configuration includes over committing the storage-related resources of the distributed storage system. 
     
     
         8 . The computer node of  claim 1 , wherein said UDSP agent is further configured, in response to a failure to determine said configuration, to provide a user with a failure notification or with a recommendation indicative of an addition of at least one additional storage-related resource which will allow successful calculation of a configuration. 
     
     
         9 . A method of operating a computer node configured to being connected to an infrastructure layer of a distributed storage system, said infrastructure layer including interconnected computer nodes, at least one of said interconnected computer nodes comprising one or more storage-related resources, the method comprising:
 receiving at least one Service Level Specification (SLS) comprising user-defined storage requirements referring to at least one logical storage entity, and storage-related resources parameters data relating to one or more storage-related resources connected to said interconnected computer nodes;   calculating a configuration for the distributed storage system based, at least, on said at least one SLS and said storage-related resources parameters data;   automatically allocating at least part of one of said storage-related resources according to the calculated configuration;   receiving dynamic behavior parameters data relating to dynamic behavior of the distributed storage system;   upon at least one SLS being breached, calculating a reconfiguration for the distributed storage system, based, at least, on said at least one SLS, said storage-related resources parameters data and said dynamic behavior parameters data; and   automatically allocating at least part of one of said storage-related resources according to the calculated reconfiguration.   
     
     
         10 . The method of  claim 9  further comprising:
 receiving a task comprising at least one assignment; 
 calculating grades for at least one of said assignments in respect of at least one of said interconnected computer nodes, such that each grade being indicative of a suitability of a respective computer node of said interconnected computer nodes to execute a respective assignment of said assignments while meeting at least one Service Level Specification (SLS) requirement, said grade is calculated also based on parameters data relating to one or more storage-related resources connected to said respective computer node, if any; 
 executing one or more of said assignments or route the task to a more suitable computer node, based on the calculated grades; 
 calculating grades for assignments of following tasks in respect of at least one added interconnected computer node of an updated infrastructure layer; said updated infrastructure layer is created in response to adding at least one additional interconnected computer node thereto; and 
 executing one or more of said assignments of following tasks or routing said following tasks to a more suitable computer node of said updated infrastructure layer, based on the calculated grades. 
 
     
     
         11 . The method of  claim 9 , wherein said calculating a configuration is performed by an optimization engine. 
     
     
         12 . The method of  claim 9 , wherein said at least one user-defined storage-related requirement includes at least one of:
 a. location;   b. local protection level;   c. backup retention policy;   d. remote protection level;   e. performance levels   f. encryption level;   g. de-duplication;   h. compression;   i. storage method.   
     
     
         13 . The method of  claim 9 , wherein said one or more storage-related resources include at least one storage-related resource of the following storage-related resource categories:
 cache resources, data storage resources and network resources.   
     
     
         14 . The method of  claim 9 , wherein said storage-related resources are distributed among at least two locations. 
     
     
         15 . The method of  claim 9 , wherein said configuration includes over committing the storage-related resources of the distributed storage system. 
     
     
         16 . The method of  claim 9  further comprising:
 in response to a failure to determine said configuration, providing a user with a failure notification or with a recommendation indicative of an addition of at least one additional resource which will allow successful calculation of a configuration. 
 
     
     
         17 . A distributed storage system comprising:
 an infrastructure layer including interconnected computer nodes, wherein:   each one of said interconnected computer nodes comprising at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent, wherein said UDSP agent is configured to:   receive at least one Service Level Specification (SLS) comprising user-defined storage requirements referring to at least one logical storage entity, and storage-related resources parameters data relating to one or more storage-related resources connected to said interconnected computer nodes;   calculate a configuration for the distributed storage system based, at least, on said at least one SLS and said storage-related resources parameters data;   automatically allocate at least part of one of said storage-related resources according to the calculated configuration;   receive dynamic behavior parameters data relating to dynamic behavior of the distributed storage system;   upon at least one SLS being breached, calculate a reconfiguration for the distributed storage system, based, at least, on said at least one SLS, said storage-related resources parameters data and said dynamic behavior parameters data; and   automatically allocate at least part of one of said storage-related resources according to the calculated reconfiguration.   
     
     
         18 . The distributed storage system of  claim 17 , wherein said UDSP agent is further configured to:
 receive a task comprising at least one assignment;
 calculate grades for at least one of said assignments in respect of at least one of said interconnected computer nodes, such that each grade being indicative of a suitability of a respective computer node of said interconnected computer nodes to execute a respective assignment of said assignments while meeting at least one user-defined storage requirement of said at least one SLS, said grade is calculated also based on parameters data relating to one or more storage-related resources connected to said respective computer node, if any; 
 execute one or more of said assignments or route the task to a more suitable computer node, based on the calculated grades; 
 calculate grades for assignments of following tasks in respect of at least one added interconnected computer node of an updated infrastructure layer; said updated infrastructure layer is created in response to adding at least one additional interconnected computer node thereto; and 
 execute one or more of said assignments of following tasks or route said following tasks to a more suitable computer node of said updated infrastructure layer, based on the calculated grades. 
   
     
     
         19 . The distributed storage system of  claim 17 , wherein said configuration is calculated by an optimization engine. 
     
     
         20 . The distributed storage system of  claim 17 , wherein said at least one user-defined storage-related requirement includes at least one of:
 a. location;   b. local protection level;   c. backup retention policy;   d. remote protection level;   e. performance levels   f. encryption level;   g. de-duplication;   h. compression;   i. storage method.   
     
     
         21 . The distributed storage system of  claim 17 , wherein said one or more storage-related resources include at least one storage-related resource of the following storage-related resource categories: cache resources, data storage resources and network resources. 
     
     
         22 . The distributed storage system of  claim 17 , wherein said storage-related resources are distributed among at least two locations. 
     
     
         23 . The distributed storage system of  claim 17 , wherein said configuration includes over committing the storage-related resources of the distributed storage system. 
     
     
         24 . The distributed storage system of  claim 17 , wherein said UDSP agent is further configured, in response to a failure to determine said configuration, to provide a user with a failure notification or with a recommendation indicative of an addition of at least one additional resource which will allow successful calculation of a configuration. 
     
     
         25 . A distributed storage system comprising:
 an infrastructure layer including interconnected computer nodes, wherein:   each one of said interconnected computer nodes comprising at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent, wherein said UDSP agent is configured to:   receive at least one Service Level Specification (SLS) comprising user-defined storage requirements referring to at least one logical storage entity, and storage-related resources parameters data relating to one or more storage-related resources connected to said interconnected computer nodes;   calculate a configuration for the distributed storage system based, at least, on said at least one SLS and said storage-related resources parameters data; and   automatically allocate at least part of one of said storage-related resources according to the calculated configuration.   
     
     
         26 . A computer node configured to being connected to an infrastructure layer of a distributed storage system, said infrastructure layer including interconnected computer nodes, the computer node comprising:
 at least one processing resource configured to execute a Unified Distributed Storage Platform (UDSP) agent, wherein said UDSP agent is configured to:   receive at least one Service Level Specification (SLS) comprising user-defined storage requirements referring to at least one logical storage entity, and storage-related resources parameters data relating to one or more storage-related resources connected to said interconnected computer nodes;   calculate a configuration for the distributed storage system based, at least, on said at least one SLS and said storage-related resources parameters data; and   automatically allocate at least part of one of said storage-related resources according to the calculated configuration.   
     
     
         27 . A method of operating a computer node configured to being connected to an infrastructure layer of a distributed storage system, said infrastructure layer including interconnected computer nodes, the method comprising:
 receiving at least one Service Level Specification (SLS) comprising user-defined storage requirements referring to at least one logical storage entity, and storage-related resources parameters data relating to one or more storage-related resources connected to said interconnected computer nodes;   calculating a configuration for the distributed storage system based, at least, on said at least one SLS and said storage-related resources parameters data; and   automatically allocating at least part of one of said storage-related resources according to the calculated configuration.