Traffic and load aware dynamic queue management

Abstract

Some embodiments provide a queue management system that efficiently and dynamically manages multiple queues that process traffic to and from multiple virtual machines (VMs) executing on a host. This system manages the queues by (1) breaking up the queues into different priority pools with the higher priority pools reserved for particular types of traffic or VM (e.g., traffic for VMs that need low latency), (2) dynamically adjusting the number of queues in each pool (i.e., dynamically adjusting the size of the pools), (3) dynamically reassigning a VM to a new queue based on one or more optimization criteria (e.g., criteria relating to the underutilization or overutilization of the queue).

We claim: 
     
         1 . For an electronic device that comprises a network interface card (NIC) with a plurality of queues, a method of managing the queues, the method comprising:
 monitoring data traffic to or from the NIC;   based on the monitoring, specifying a pool and assigning a set of the queues to the pool, said pool having a set of criteria for managing data traffic through the set of queues; and   directing a subset of the data traffic to the set of queues based on the set of criteria.   
     
     
         2 . The method of  claim 1 , wherein the pool is a first pool, the set of queues is a first set of queues, the set of criteria is a first set of criteria, and the subset of data traffic is a first subset of data traffic, the method comprising:
 based on the monitoring, specifying a second pool and assigning a second set of the queues to the second pool, said second pool having a second set of criteria for managing data traffic through the second set of queues; and   directing a second subset of the data traffic to the second set of queues based on the second set of criteria;   wherein the first set of criteria differs from the second set of criteria.   
     
     
         3 . The method of  claim 2 , wherein each particular pool's set of criteria specifies a maximum threshold amount of data traffic for passing through each queue of the particular pool, wherein the first set of criteria specifies a different maximum threshold than the second set of criteria. 
     
     
         4 . The method of  claim 3  further comprising assigning a new queue to a particular pool's set of queues when data traffic through at least a subset of queues in the particular pool exceeds the maximum threshold amount. 
     
     
         5 . The method of  claim 4 , wherein the subset of queues of the particular pool includes all the queues in the particular pool's set of queues. 
     
     
         6 . The method of  claim 4 , wherein the subset of queues of the particular pool does not include all the queues in the particular pool's set of queues. 
     
     
         7 . The method of  claim 2 , wherein each particular pool's sets of criteria specifies a minimum threshold amount of data traffic for passing through each queue of the particular pool, wherein the first set of criteria specifies a different minimum threshold than the second set of criteria. 
     
     
         8 . The method of  claim 7  further comprising removing a particular queue from a particular pool's set of queues when data traffic through particular queue is below the minimum threshold amount for a duration of time. 
     
     
         9 . The method of  claim 2 , wherein:
 each particular pool's sets of criteria specifies a threshold amount of data traffic for passing through each queue of the particular pool,   the first set of criteria specifies a maximum threshold amount but not a minimum threshold amount, while the second set of criteria specifies a minimum threshold amount but not a maximum threshold amount.   
     
     
         10 . The method of  claim 1 , wherein monitoring data traffic comprises monitoring data traffic associated with addressable nodes executing on the electronic device. 
     
     
         11 . The method of  claim 1 , wherein the set of queues only includes one queue. 
     
     
         12 . The method of  claim 1 , wherein the set of queues include more than one queue. 
     
     
         13 . The method of  claim 1 , wherein directing the subset of data traffic comprises specifying a set of filters in the NIC to route the subset of data traffic through the set of queues. 
     
     
         14 . The method of  claim 13 , wherein the set of filters route the subset of data traffic through the set of queues to a set of addressable destination nodes executing on the electronic device. 
     
     
         15 . The method of  claim 1 , wherein directing the subset of data traffic comprises specifying a set of filters that route the subset of data traffic from a set of addressable source nodes executing on the electronic device out of the electronic device through the set of queues. 
     
     
         16 . The method of  claim 15 , wherein the set of filters is defined in a network layer that shares a set of networking resources on the electronic device with multiple addressable source nodes. 
     
     
         17 . The method of  claim 15 , wherein the set of filters is defined in a network virtualization layer that shares a set of networking resources on the electronic device with multiple addressable nodes that execute on the electronic device. 
     
     
         18 . The method of  claim 15 , wherein said monitoring, specifying and directing are operations performed by a network virtualization layer that shares a set of networking resources on the electronic device amongst multiple different virtual modules, wherein the set of filters is defined in the network virtualization layer to assign data traffic from different virtual modules to different queues in the plurality of queues. 
     
     
         19 . For an electronic device that comprises a network interface card (NIC) with a plurality of queues, a method of managing the queues, the method comprising:
 specifying a default pool comprising at least a first queue;   assigning a set of data traffic to the default pool;   determining that a non-default pool should be specified to handle a first subset of the set of data traffic;   specifying the non-default pool and a second queue to the non-default pool; and   assigning the subset of the data traffic to the second queue.   
     
     
         20 . The method of  claim 19  further comprising monitoring the set of data traffic to or from the NIC, wherein said determining is based on said monitoring. 
     
     
         21 . The method of  claim 20  further comprising:
 based on the monitoring, identifying a second subset of the set of data traffic to assign to the non-default pool; and 
 assigning the second subset to the non-default pool. 
 
     
     
         22 . The method of  claim 21 , wherein assigning the second subset comprises assigning the second subset to the second queue. 
     
     
         23 . The method of  claim 21 , wherein assigning the second subset comprises assigning the second subset to a third queue that is assigned to the non-default pool. 
     
     
         24 . The method of  claim 20  further comprising:
 monitoring the first subset of data traffic; and 
 based on the monitoring, re-assigning the first subset of data traffic back to the default pool. 
 
     
     
         25 . The method of  claim 24 , wherein the re-assigning the first subset comprises re-assigning the first subset back to the first queue. 
     
     
         26 . The method of  claim 24 , wherein the re-assigning the first subset comprises re-assigning the first subset back to a third queue that is assigned to the default pool. 
     
     
         27 . The method of  claim 19 , wherein the different pools have different sets of criteria for assigning data traffic to queues in the pools. 
     
     
         28 . The method of  claim 27 , wherein the sets of criteria for different pools differ in specifying different maximum threshold amount of data traffic for passing through each queue of the particular pool. 
     
     
         29 . The method of  claim 27 , wherein the sets of criteria for different pools differ in specifying different minimum threshold amount of data traffic for passing through each queue of the particular pool. 
     
     
         30 . The method of  claim 27 , wherein the sets of criteria for different pools differ in specifying different threshold amount of data traffic for passing through each queue of the particular pool, wherein one pool specifies a maximum threshold while another pool specifies a minimum threshold. 
     
     
         31 . The method of  claim 19 , wherein the set of data traffic comprises data traffic associated with addressable nodes executing on the electronic device. 
     
     
         32 . The method of  claim 31 , wherein the addressable nodes comprises at least one of a virtual machine (VM), a storage volume mounter, and a VM migrator. 
     
     
         33 . A method of managing a plurality of queues in a network interface of a host that executes a plurality of virtual machines (VMs), the method comprising:
 specifying first and second pools that define first and second groupings of at least a subset of the queues;   based on a first set of criteria, specifying an allocation of a first set of VMs to the queues in the first pool; and   based on a second set of criteria, specifying an allocation of a second set of VMs to the queues in the second pool, wherein the first and second set of criteria are different.   
     
     
         34 . The method of  claim 33 ,
 wherein specifying the allocation of the first set of VMs comprises using a first set of processes to specify the allocation;   wherein specifying the allocation of the second set of VMs comprises using a second set of processes to specify the allocation, wherein the first and second set of processes are different processes.