Systems and methods for content-aware load balancing

Abstract

Improved load balancing techniques are disclosed. For example, in one illustrative aspect of the invention, a method of satisfying requests in a system comprised of a plurality of servers comprises the following steps. At least one load balancer is provided for routing requests to the plurality of servers. At the at least one load balancer, a request sent from a client is obtained. At the at least one load balancer, the request is examined. Costs of satisfying the request by at least two of the plurality of servers are estimated. The estimation is based on at least one of a number and a cost of at least one remote access for satisfying the request. The request is routed to a server of the plurality of servers with a low estimated cost of satisfying the request.

1 . A method for satisfying requests in a system comprised of a plurality of servers, comprising the steps of: 
 providing at least one load balancer for routing requests to the plurality of servers;    obtaining, at the at least one load balancer, a request from a client;    examining, at the at least one load balancer, the request;    estimating costs of satisfying the request by at least two of the plurality of servers based on estimating at least one of a number and a cost of at least one remote access for satisfying the request; and    routing the request to a server of the plurality of servers with a low estimated cost of satisfying the request.    
   
   
       2 . The method of  claim 1 , wherein the step of routing is accomplished by classifying a request into a partition and routing the request to a server hosting the partition.  
   
   
       3 . The method of  claim 1 , wherein, in the step of providing at least one load balancer, the at least one load balancer comprises at least one content-unaware load balancer routing requests to a plurality of content-aware load balancers.  
   
   
       4 . The method of  claim 3 , wherein, in the step of providing at least one load balancer, at least one of the plurality of content-aware load balancers resides on at least one of the plurality of servers.  
   
   
       5 . The method of  claim 1 , wherein the step of estimating costs further comprises the steps of: 
 examining at least one parameter included in the request; and    using information about how data are partitioned among the plurality of servers to estimate at least one of numbers and costs of remote accesses for satisfying the request.    
   
   
       6 . The method of  claim 1 , wherein the step of obtaining a request further comprises obtaining a request that is sent using the Transmission Control Protocol/Internet Protocol and wherein the step of examining further comprises accepting a TCP connection.  
   
   
       7 . The method of  claim 1 , wherein the step of estimating costs comprises estimating at least one of a measure of resource utilization and a desired service level for satisfying the request.  
   
   
       8 . The method of  claim 1 , wherein the step of estimating costs may vary with respect to at least one of a number of servers and time.  
   
   
       9 . The method of  claim 1 , wherein, in the step of obtaining, at the at least one load balancer, a request from a client, the request comprises a request for dynamic data.  
   
   
       10 . A method of satisfying requests in a system comprised of a plurality of servers, comprising the steps of: 
 providing at least one load balancer for routing requests to the plurality of servers;    obtaining, at the at least one load balancer, a request from a client;    examining, at the at least one load balancer, the request;    estimating costs of satisfying the request by at least two of the plurality of servers;    sending, from the load balancer, information obtained in the estimating step to a server of the plurality of servers along with the request; and    using, at the server, the information to satisfy the request.    
   
   
       11 . A system for satisfying requests from at least one client, comprising: 
 a plurality of servers;    at least one content-aware load balancer for routing requests to the plurality of servers; and    at least one cost analyzer associated with the at least one content-aware load balancer for estimating costs of satisfying requests by different servers based on at least one of numbers and costs of remote accesses for satisfying requests.    
   
   
       12 . The system of  claim 11 , wherein the at least one content-aware load balancer comprises a plurality of content-aware load balancers and wherein the system further comprises at least one content-unaware load balancer for routing requests to the plurality of content-aware load balancers.  
   
   
       13 . The system of  claim 12 , wherein at least one of the plurality of content-aware load balancers resides on at least one of the plurality of servers.  
   
   
       14 . The system of  claim 11 , wherein a request is routed by classifying the request into a partition and routing the request to a server hosting the partition.  
   
   
       15 . The system of  claim 11 , wherein the at least one cost analyzer estimates costs by examining at least one parameter included in the request, and using information about how data are partitioned among the plurality of servers to estimate at least one of numbers and costs of remote accesses for satisfying the request.  
   
   
       16 . The system of  claim 11 , wherein a request is sent using the Transmission Control Protocol/Internet Protocol and a TCP connection is accepted when the request is examined.  
   
   
       17 . The system of  claim 11 , wherein the at least one cost analyzer estimates at least one of a measure of resource utilization and a desired service level for satisfying the request.  
   
   
       18 . The system of  claim 11 , wherein estimating costs may vary with respect to at least one of a number of servers and time.  
   
   
       19 . Apparatus for satisfying requests from at least one client in a system comprised of a plurality of servers, the apparatus comprising: 
 a memory; and    at least one processor coupled to the memory and operative to: (i) obtain a request from a client; (ii) examine content of the request; and (iii) estimate costs of satisfying the request by at least two of the plurality of servers based on estimating at least one of a number and a cost of at least one remote access for satisfying the request.    
   
   
       20 . A method for satisfying requests from at least one client in a system comprised of a plurality of servers, comprising the steps of: 
 obtaining a request from a client;    examining content of the request; and    estimating costs of satisfying the request by at least two of the plurality of servers based on estimating at least one of a number and a cost of at least one remote access for satisfying the request.    
   
   
       21 . An article of manufacture for use in satisfying requests from at least one client in a system comprised of a plurality of servers, comprising a machine readable medium containing one or more programs which when executed implement the steps of: 
 obtaining a request from a client;    examining content of the request; and    estimating costs of satisfying the request by at least two of the plurality of servers based on estimating at least one of a number and a cost of at least one remote access for satisfying the request.    
   
   
       22 . A method for providing a load balancing service, comprising the step of: 
 a service provider providing a service to a customer which comprises:    obtaining a request from a client;    examining content of the request; and    estimating costs of satisfying the request by at least two of the plurality of servers based on estimating at least one of a number and a cost of at least one remote access for satisfying the request.