On-Demand Compute Environment

Abstract

An on-demand compute environment comprises a plurality of nodes within an on-demand compute environment available for provisioning and a slave management module operating on a dedicated node within the on-demand compute environment, wherein upon instructions from a master management module at a local compute environment, the slave management module modifies at least one node of the plurality of nodes.

1 - 18 . (canceled) 
     
     
         19 . A method of managing a first compute environment, the method comprising:
 receiving, at a first software module operating in the first compute environment, from a second software module associated with an external compute environment separate from the first compute environment, a specification of resources to be used as part of processing of a workload, the specification of resources comprising both (i) first data explicitly specifying resources, the specified resources which must be provided by the first compute environment when servicing the workload, and (ii) second data specifying one or more flexible resources, the flexible resources which can be dynamically modified by at least the first compute environment;   creating a new software module within the first compute environment;   provisioning at least one node in the first compute environment to perform at least part of the processing of the workload, wherein the new software module is configured to manage the provisioned at least one node and communicate with the second software module; and   removing the new software module after one or more removal criteria have been met.   
     
     
         20 . The method of  claim 19 , wherein:
 the new software module is exclusively associated with the workload; and   the one or more removal criteria comprise completion of the at least part of the processing of the workload.   
     
     
         21 . The method of  claim 19 , wherein the second data specifying the one or more flexible resources is based at least in part on one or more attributes of the external compute environment. 
     
     
         22 . The method of  claim 19 , wherein the provisioning of the at least one node is based at least in part a computerized analysis of at least part of the workload. 
     
     
         23 . The method of  claim 19 , wherein the specification of resources specifies an allocation of time, and wherein the method further comprises automatically adjusting a duration of the allocation of time. 
     
     
         24 . The method of  claim 23 , wherein the automatically adjusting a duration of the allocation of time comprises automatically adjusting based at least on at least one of (i) accommodating provisioning overhead, or (ii) one or more clean up processes. 
     
     
         25 . The method of  claim 19 , wherein:
 the external compute environment comprises a local compute environment, and the first compute environment comprises an on-demand compute environment that is separately managed from the local compute environment and configured to be utilized when the local compute environment has insufficient resources to process workload submitted to the local computer environment; and   the on-demand compute environment and the local compute environment are in data communication with one another via at least one wireless or wireline communication network.   
     
     
         26 . The method of  claim 19 , where in the provisioning further comprises creating at least one virtual cluster within the first compute environment, the creation of the at least one virtual cluster comprising dynamic selection of a plurality of compute resources, and provisioning of the dynamically selected compute resources for use by at least part of the workload. 
     
     
         27 . The method of  claim 19 , wherein the provisioning comprises, responsive to the receiving the specification of resources, causing (i) dynamic co-allocation of a plurality of compute resources, and (ii) provisioning of the co-allocated compute resources for use by at least part of the workload. 
     
     
         28 . The method of  claim 19 , wherein the creating a new software in the first compute environment comprises creating one or more new software instances on one or more respective ones of a plurality of compute nodes within the first compute environment. 
     
     
         29 . The method of  claim 19 , wherein:
 the receiving the specification of resources is in preparation for an assumption of at least a portion of the workload by the first compute environment from the external compute environment; and   the second data specifying one or more flexible resources comprises data relating to a network storage device requirement.   
     
     
         30 . The method of  claim 19 , wherein:
 the receiving the specification of resources is in preparation for an assumption of at least a portion of the workload by the first compute environment from the external compute environment; and   the second data specifying one or more flexible resources comprises data relating to at least one of a operating system (OS) or installed software application.   
     
     
         31 . The method of  claim 19 , wherein the provisioning the at least one node in the first compute environment to perform at least part of the processing of the workload comprises using at least the first data and the second data. 
     
     
         32 . The method of  claim 31 , wherein at least one of the first data or the second data is based at least in part on one or more queue attributes. 
     
     
         33 . The method of  claim 19 , further comprising (i) identifying one or more resource constraints associated with processing the workload within the first compute environment, and (ii) causing dynamic modification of at least a portion of the first compute environment based at least on the identified one or more resource constraints. 
     
     
         34 . The method of  claim 33 , wherein:
 the identifying the one or more resource constraints comprises identifying one or more resource provisioning requirements necessary for processing of the workload; and   the provisioning of the at least one node comprises provisioning based at least on the one or more resource provisioning requirements.   
     
     
         35 . The method of  claim 19 , wherein the receiving the specification of resources to process the workload comprises receiving as specification of resources that is configured based at least on: (i) notification of a user submitting the workload for processing that additional compute resources are required to process the workload, and (ii) affirmative action by the notified user acknowledging or requesting the additional compute resources. 
     
     
         36 . The method of  claim 19 , wherein the receiving the specification of resources to process the workload comprises receiving as specification of resources that (i) is created automatically such that it requires no user intervention, and (ii) is configured based at least on determination that compute resources in addition to those of the external compute environment are required to process the workload. 
     
     
         37 . The method of  claim 19 , wherein:
 the first compute environment comprises one or more untrusted resources; and   the method further comprises receiving data from the external compute environment relating to one or more security models, the one or more security models configured to maintain a level of security even with the presence of the one or more untrusted resources within the compute environment.   
     
     
         38 . A computerized apparatus configured for communication with a first compute environment, the computerized apparatus associated with an external compute environment separate from the first compute environment, the computerized apparatus comprising:
 a processor; and   a storage device in data communication with the processor, the storage device comprising a non-transitory computer-readable storage medium having stored therein instructions which, when executed by the processor, cause the computerized apparatus to:
 receive data relating to at least one rigid resource requirement for processing workload; 
 cause creation of a specification of resources to process the workload, the specification of resources comprising both (i) the data relating to the at least one rigid resource requirement, and (ii) data relating to at least one flexible resource requirement; 
 cause transmission of the specification of resources from a first software module of the computerized apparatus to a second software module operating on at least one node in the first compute environment, the specification of resources requiring creation of a third software module on at least one node in the first compute environment, the third software module associated uniquely with the specification of resources and configured to manage at least one compute resource in the first compute environment to process at least part of the workload. 
   
     
     
         39 . The computerized apparatus of  claim 38 , wherein the third software module is configured for removal from the first compute environment based at least on completion of processing of the workload. 
     
     
         40 . The computerized apparatus of  claim 39 , wherein the third software module being uniquely associated with the specification of resources enables the removal of the third software module after the completion of processing of the workload. 
     
     
         41 . The computerized apparatus of  claim 38 , wherein the at least one flexible resource requirement is based at least on one or more attributes of the external compute environment, the one or more attributes selected from the group consisting of: (i) operating system configuration, and (ii) one or more installed software applications. 
     
     
         42 . The computerized apparatus of  claim 38 , wherein the at least one rigid resource requirement is selected from the group consisting of: (i) processor architecture, and (ii) RAM (random access memory) configuration. 
     
     
         43 . The computerized apparatus of  claim 38 , wherein the external compute environment comprises a compute environment having a plurality of commonly managed compute resources, and the first compute environment comprises an on-demand compute environment that is separately managed from the local compute environment but in data communication with the external compute environment via a data network. 
     
     
         44 . The computerized apparatus of  claim 38 , wherein the specification of resources is configured to enable creation of at least one virtual cluster within the first compute environment, the creation of the at least one virtual cluster comprising dynamic selection of a plurality of compute resources, and provisioning of the selected compute resources for use by at least part of the workload as part of the processing thereof. 
     
     
         45 . The computerized apparatus of  claim 38 , wherein at least one of the second software module or the third software module are configured to dynamically modify the at least one compute resource in the first compute environment based at least on the data relating to the flexible resource requirement. 
     
     
         46 . The computerized apparatus of  claim 38 , wherein the instructions are further configured to, when executed by the processor, cause the computerized apparatus to (i) identify one or more resource constraints associated with processing the workload within the first compute environment, and (ii) cause dynamic modification of at least a portion of the first compute environment based at least on a) the identified one or more resource constraints and b) the data relating to the at least one flexible resource requirement. 
     
     
         47 . The computerized apparatus of  claim 46 , wherein the identification of the one or more resource constraints comprises identification of one or more resource provisioning requirements necessary for processing of the workload either a) by or at a prescribed time, or b) within a prescribed period or duration of time. 
     
     
         48 . The computerized apparatus of  claim 38 , wherein the receipt of the data relating to the at least one rigid resource requirement comprises receipt via a computerized job request to the external compute environment, the received data requiring at least one of computer processor or memory requirements. 
     
     
         49 . The computerized apparatus of  claim 38 , wherein the computerized apparatus comprises a security model configured to account for addition of one or more untrusted resources to at least one of the external compute environment or the first compute environment. 
     
     
         50 . The computerized apparatus of  claim 38 , wherein the causation of creation of the specification of resources to process the workload is based at least in part on exceeding one or more trigger criteria or events associated with the external compute environment. 
     
     
         51 . The computerized apparatus of  claim 50 , wherein the one or more trigger criteria or events associated with the external compute environment comprise at least one of a) a resource failure within the external compute environment; or b) exceeding a resource consumption threshold associated with the external compute environment. 
     
     
         52 . Storage apparatus comprising a non-transitory computer-readable storage medium having stored therein instructions which, when executed by a processor apparatus of a computerized apparatus within a first compute environment, cause the computerized apparatus to perform operations comprising:
 receive data relating to at least one workload to be processed;   based at least on an evaluation of the received data, cause creation of a specification of resources to process at least a portion of the at least one workload, the specification of resources comprising both (i) first data relating to the at least one explicit resource requirement, and (ii) second data relating to at least one second resource requirement, the at least one second resource requirement based at least on one or more attributes of the first compute environment;   cause transmission of the specification of resources from a first module of the computerized apparatus to a second module operating in a second compute environment, the specification of resources configured to cause creation of a new software instance in the second compute environment, the new software instance a) particularly associated with the at least one workload, and b) configured to communicate with at least one compute resource in the second compute environment to process the at least one workload.   
     
     
         53 . The storage apparatus of  claim 52 , wherein the instructions are further configured to, when executed, receive data communications at the first software module from the second compute environment indicating that the new software instance has been removed from the second compute environment after the at least one workload has been processed. 
     
     
         54 . A method of operating a first compute environment to cause utilization of compute resources within a second, separately managed compute environment, the method comprising:
 receiving at least one first data structure at the first compute environment, the at least one data structure comprising a request to process workload, the request comprising at least one explicit resource requirement;   creating at least one second data structure, the at least one second data structure comprising both (i) data relating to the at least one explicit resource requirement, and (ii) data relating to at least one additional resource requirement, the at least one additional resource requirement based at least on at least one of a network attribute or operating system attribute of the first compute environment;   causing transmission, from a first software module of the first compute environment to a second software module operating in the second compute environment, the at least one second data structure, the transmission configured to cause creation of a new software instance in the second compute environment, the new software instance configured to communicate with at least one compute node in the second compute environment to effect processing of at least a portion of the workload; and   receiving at least one data communication at the first compute environment from the second compute environment, the at least one data communication indicating that the new software instance has been removed from the second compute environment pursuant to completion of processing of the at least portion of the workload.