Systems and methods for rate-based load balancing

Abstract

According to examples, a system for rate-based load balancing may include a processor and a memory storing instructions. The processor may, through execution of the instructions, cause the system to receive a request for processing. The system may further identify a target server to transmit the request using a rate-based load balancing technique. In some examples, the rate-based load balancing technique may include: selecting a server, from a plurality of servers, as a potential target; receiving a readiness indicator for the selected server; and designating the selected server as the target server based on the readiness indicator. The system may transmit the request to the target server for processing.

1 - 20 . (canceled) 
     
     
         21 . A system, comprising:
 a processor; and   a memory storing instructions, which when executed by the processor, cause the processor to:
 select a server, from a plurality of servers, as a potential target; 
 receive a readiness indicator for the selected server, wherein the readiness indicator is based on a timing technique using a fixed-capacity token bucket and a classification algorithm to determine if the selected server is to be designated as a target server; and 
 designate the selected server as the target server based on the readiness indicator; and 
 transmit a request to the target server for processing. 
   
     
     
         22 . The system of  claim 21 , wherein the timing technique measures and identifies whether a predetermined time period has passed. 
     
     
         23 . The system of  claim 22 , wherein a rate controller associated with the selected server generates the readiness indicator based on the timing technique, wherein the readiness indicator is a digital transmission that indicates whether the selected server is able to process the request. 
     
     
         24 . The system of  claim 23 , wherein the rate controller provides the readiness indicator upon selection of the server, by the processor, as the potential target or following passage of the predetermined time period. 
     
     
         25 . The system of  claim 24 , wherein the predetermined time period is configured by a user. 
     
     
         26 . The system of  claim 24 , wherein the predetermined time period is configured based on data associated with at least the request or the plurality of servers. 
     
     
         27 . The system of  claim 24 , wherein the predetermined time period is configured by using an artificial intelligence (AI) based machine learning technique based on information associated with the request or the selected server, wherein the artificial intelligence (AI) based machine learning technique utilizes at least one of a neural network, a tree-based model, a Bayesian network, a support vector, clustering, a kernel method, a spline, or a knowledge graph. 
     
     
         28 . The system of  claim 21 , wherein the classification algorithm provides assignment of instances to pre-defined classes to decide whether there are matches or correlation and into which one or more tokens, each comprising a packet of predetermined size, are added into the fixed-capacity token bucket at a predetermined fixed rate of lir seconds, where r is an integer, such that, in the event a token arrives when the fixed-capacity token bucket is full, the token is discarded and the fixed-capacity token bucket remains full until the selected server is designated as the target server based on the readiness indicator. 
     
     
         29 . The system of  claim 21 , wherein the system is an online system for processing digital requests associated with at least one of advertisements, payments, or social media. 
     
     
         30 . A method, comprising:
 selecting a server, from a plurality of servers, as a potential target;   receiving a readiness indicator for the selected server, wherein the readiness indicator is based on a timing technique using a fixed-capacity token bucket and a classification algorithm to determine if the selected server is to be designated as a target server; and   designating the selected server as the target server based on the readiness indicator; and   transmitting a request to the target server for processing.   
     
     
         31 . The method of  claim 30 , wherein the timing technique measures and identifies whether a predetermined time period has passed. 
     
     
         32 . The method of  claim 31 , wherein a rate controller associated with the selected resource generates the readiness indicator based on the timing technique, wherein the readiness indicator is a digital transmission that indicates whether the selected resource is able to process the request. 
     
     
         33 . The method of  claim 32 , wherein the rate controller provides the readiness indicator upon selection of the resource, by the processor, as the potential target or following termination of a predetermined time period. 
     
     
         34 . The method of  claim 33 , wherein the predetermined time period is configured by a user. 
     
     
         35 . The method of  claim 33 , wherein the predetermined time period is configured based on data associated with at least the request or the plurality of resources. 
     
     
         36 . The method of  claim 33 , wherein the predetermined time period is configured by using an artificial intelligence (AI) based machine learning technique based on information associated with the request or the selected resource, wherein the artificial intelligence (AI) based machine learning technique utilizes at least one of a neural network, a tree-based model, a Bayesian network, a support vector, clustering, a kernel method, a spline, or a knowledge graph. 
     
     
         37 . The method of  claim 30 , wherein the classification algorithm provides assignment of instances to pre-defined classes to decide whether there are matches or correlation and into which one or more tokens, each comprising a packet of predetermined size, are added into the fixed-capacity token bucket at a predetermined fixed rate of 1/r seconds, where r is an integer, such that, in the event a token arrives when the fixed-capacity token bucket is full, the token is discarded and the fixed-capacity token bucket remains full until the selected server is designated as the target server based on the readiness indicator. 
     
     
         38 . The method of  claim 30 , wherein the rate-based load balancing system is an online system for processing digital requests associated with at least one of advertisements, payments, or social media. 
     
     
         39 . A non-transitory computer-readable storage medium having an executable stored thereon, which when executed instructs a processor to:
 selecting a server, from a plurality of servers, as a potential target;   receiving a readiness indicator for the selected server, wherein the readiness indicator is based on a timing technique using a fixed-capacity token bucket and a classification algorithm to determine if the selected server is to be designated as a target server; and   designating the selected server as the target server based on the readiness indicator; and   transmitting a request to the target server for processing.   
     
     
         40 . The non-transitory computer-readable storage medium of  claim 39 , wherein the readiness indicator is provided by a rate controller associated with the selected resource following passage of a predetermined time period.