Accelerator or accelerated functions as a service using networked processing units

Abstract

Various approaches for deploying and controlling distributed accelerated compute operations with the use of infrastructure processing units (IPUs) and similar networked processing units are disclosed. A system for orchestrating acceleration functions in a network compute mesh is configured to access a flowgraph, the flowgraph including data producer-consumer relationships between a plurality of tasks in a workload; identify available artifacts and resources to execute the artifacts to complete each of the plurality of tasks, wherein an artifact is an instance of a function to perform a task of the plurality of tasks; determine a configuration assigning artifacts and resources to each of the plurality of tasks in the flowgraph; and schedule, based on the configuration, the plurality of tasks to execute using the assigned artifacts and resources.

What is claimed is: 
     
         1 . A system for orchestrating acceleration functions in a network compute mesh, comprising:
 a memory device configured to store instructions; and   a processor subsystem, which when configured by the instructions, is operable to:
 access a flowgraph, the flowgraph including data producer-consumer relationships between a plurality of tasks in a workload; 
 identify available artifacts and resources to execute the artifacts to complete each of the plurality of tasks, wherein an artifact is an instance of a function to perform a task of the plurality of tasks; 
 determine a configuration assigning artifacts and resources to each of the plurality of tasks in the flowgraph; and 
 schedule, based on the configuration, the plurality of tasks to execute using the assigned artifacts and resources. 
   
     
     
         2 . The system of  claim 1 , wherein the artifact comprises a bitstream to program a field-programmable gate array (FPGA). 
     
     
         3 . The system of  claim 1 , wherein the artifact comprises an executable file to execute on a central processing unit (CPU). 
     
     
         4 . The system of  claim 1 , wherein the artifact comprises a binary file to configure a coarse-grained reconfigurable array (CGRA). 
     
     
         5 . The system of  claim 1 , wherein the resources comprise a central processing unit. 
     
     
         6 . The system of  claim 1 , wherein the resources comprise a network-accessible processing unit. 
     
     
         7 . The system of  claim 1 , wherein the resources comprise a graphics processing unit. 
     
     
         8 . The system of  claim 1 , wherein the resources comprise an application specific integrated circuit (ASIC). 
     
     
         9 . The system of  claim 1 , wherein the resources comprise a field-programmable gate array (FPGA). 
     
     
         10 . The system of  claim 1 , wherein the resources comprise a coarse-grained reconfigurable array (CGRA). 
     
     
         11 . The system of  claim 1 , wherein determining the configuration comprises:
 analyzing a service level objective (SLO); and   assigning artifacts and resources to each of the plurality of tasks to satisfy the SLO.   
     
     
         12 . The system of  claim 1 , wherein determining the configuration comprises performing one or more of minimizing an amount of data movement between the plurality of tasks and a storage device, minimizing latency of workload execution, maximizing resource utilization for the workload, maximizing capacity of acceleration available resources, minimizing the power consumption of the workload, or optimizing the perf/watt or perf/$/watt metric for the workload. 
     
     
         13 . The system of  claim 1 , wherein scheduling the plurality of tasks comprises:
 communicating from a first network-accessible processing unit to a second network-accessible processing unit via an application programming interface (API), to schedule a task of the plurality of tasks to execute using an artifact executing on a resource managed by the second network-accessible processing unit.   
     
     
         14 . The system of  claim 13 , wherein scheduling the plurality of task comprises:
 lending or transferring resources from the first network-accessible processing unit to the second network-accessible processing unit for use when executing the artifact.   
     
     
         15 . The system of  claim 1 , wherein a task of the plurality of tasks produces a data result, which is stored in a distributed databased accessible by at least one other task of the plurality of tasks. 
     
     
         16 . A method for orchestrating acceleration functions in a network compute mesh, comprising:
 accessing a flowgraph, the flowgraph including data producer-consumer relationships between a plurality of tasks in a workload;   identifying available artifacts and resources to execute the artifacts to complete each of the plurality of tasks, wherein an artifact is an instance of a function to perform a task of the plurality of tasks;   determining a configuration assigning artifacts and resources to each of the plurality of tasks in the flowgraph; and   scheduling, based on the configuration, the plurality of tasks to execute using the assigned artifacts and resources.   
     
     
         17 . The method of  claim 16 , wherein determining the configuration comprises:
 analyzing a service level objective (SLO); and   assigning artifacts and resources to each of the plurality of tasks to satisfy the SLO.   
     
     
         18 . The method of  claim 16 , wherein determining the configuration comprises performing one or more of minimizing an amount of data movement between the plurality of tasks and a storage device, minimizing latency of workload execution, maximizing resource utilization for the workload, maximizing capacity of acceleration available resources, minimizing the power consumption of the workload, or optimizing the perf/watt or perf/$/watt metric for the workload. 
     
     
         19 . The method of  claim 16 , wherein scheduling the plurality of tasks comprises:
 communicating from a first network-accessible processing unit to a second network-accessible processing unit via an application programming interface (API), to schedule a task of the plurality of tasks to execute using an artifact executing on a resource managed by the second network-accessible processing unit.   
     
     
         20 . The method of  claim 19 , wherein scheduling the plurality of task comprises:
 lending or transferring resources from the first network-accessible processing unit to the second network-accessible processing unit for use when executing the artifact.   
     
     
         21 . The method of  claim 16 , wherein a task of the plurality of tasks produces a data result, which is stored in a distributed databased accessible by at least one other task of the plurality of tasks. 
     
     
         22 . At least one machine-readable medium including instructions for orchestrating acceleration functions in a network compute mesh, which when executed by a machine, cause the machine to:
 access a flowgraph, the flowgraph including data producer-consumer relationships between a plurality of tasks in a workload;   identify available artifacts and resources to execute the artifacts to complete each of the plurality of tasks, wherein an artifact is an instance of a function to perform a task of the plurality of tasks;   determine a configuration assigning artifacts and resources to each of the plurality of tasks in the flowgraph; and   schedule, based on the configuration, the plurality of tasks to execute using the assigned artifacts and resources.   
     
     
         23 . The at least one machine-readable medium of  claim 22 , wherein the instructions to determine the configuration comprise the instructions to:
 analyze a service level objective (SLO); and   assign artifacts and resources to each of the plurality of tasks to satisfy the SLO.   
     
     
         24 . The at least one machine-readable medium of  claim 22 , wherein the instructions to determine the configuration comprises instructions to performing one or more of minimizing an amount of data movement between the plurality of tasks and a storage device, minimizing latency of workload execution, maximizing resource utilization for the workload, maximizing capacity of acceleration available resources, minimizing the power consumption of the workload, or optimizing the perf/watt or perf/$/watt metric for the workload. 
     
     
         25 . The at least one machine-readable medium of  claim 22 , wherein the instructions to schedule the plurality of tasks comprise instructions to:
 communicate from a first network-accessible processing unit to a second network-accessible processing unit via an application programming interface (API), to schedule a task of the plurality of tasks to execute using an artifact executing on a resource managed by the second network-accessible processing unit.