Self-adjusting tiered caching system to optimize traffic performance and origin offload

Abstract

A tiered caching system (e.g., in an overlay network such as CDN) wherein SWR at an edge server therein is automatically set (preferably based on request rate), while SWR at a parent server therein has its SWR set at or near 0. Collectively, these SWR settings provide for a self-adjusting tiered caching system that optimizes both end user performance and the origin traffic offload

1 . A system associated with an origin server at which content to be delivered to a set of client machines is published, comprising:
 an overlay network comprising a computing machine having a hardware processor, and computer memory;   a server process executing in the hardware processor;   the server process configured to enforce a stale-while-revalidate (SWR) cache-control whereby the server process is enabled to return a stale response in response to a request while attempting to revalidate the request in a background action;   the server process configured to compute and enforce an SWR cache-control value set to a first SWR value, wherein the first value is a function of rate of requests from the set of client machines received at the server process;   wherein the first SWR value is SWR (% of TTL)=100/(TTL*R), wherein TTL refers to a Time-To-Live (TTL) of the content and R is the rate of requests;   wherein enforcing the first SWR value improves an operation of the overlay network by reducing internal traffic being the edge server process and the origin server process.   
     
     
         2 . The system as described in  claim 1  wherein the first SWR value satisfies a constraint that is: 0<first SWR value<100. 
     
     
         3 . The system as described in  claim 1  wherein the first SWR value also satisfies a constraint that is: (first SWR value)*TTL>a minimum number of seconds. 
     
     
         4 . The system as described in  claim 3  wherein the minimum number of seconds is one of: a fixed number of seconds, and a multiplier of an observed latency associated with fetching the content from the server process. 
     
     
         5 . The system as described in  claim 1  wherein the first SWR value defines an asynchronous SWR refresh window. 
     
     
         6 . The system as described in  claim 5  wherein the asynchronous SWR refresh window starts before or at a time when the TTL expires. 
     
     
         7 . The system as described in  claim 1  wherein the first SWR value is computed dynamically. 
     
     
         8 . The system as described in  claim 1  wherein the server process comprises a component of a tiered caching hierarchy for the content. 
     
     
         9 . A server, operative in a computer network, the server being configured to enforce a stale-while-revalidate (SWR) cache-control whereby the server is enabled to return a stale response in response to a request while attempting to revalidate the request in a background action, comprising:
 a hardware processor;   computer memory holding computer program instructions executed by the hardware processor to compute and enforce an SWR cache-control value;   wherein the SWR cache-control value is computed as a function of a Time-To-Live (TTL) of the content, and a rate of requests for the content;   wherein enforcing the SWR cache-control value improves an operation of the computer network by reducing traffic between the server and an origin server at which the content is published.   
     
     
         10 . A method of self-adjusting a tiered caching system to optimize traffic performance and origin server off-load, the tiered caching system comprising an edge server and a forward server, each of the edge server and the forward server being configured to enforce a stale-while-revalidate (SWR) cache-control whereby the server is enabled to return a stale response in response to a request while attempting to revalidate the request in a background action, comprising:
 enforcing a first SWR cache control value at the edge server; and   concurrently enforcing a second SWR cache control value at the forward server;   the first SWR value being computed as a function of request rate for content at the edge server, and the second SWR value being substantially zero;   wherein enforcing the first and the second SWR cache control values improves an operation of the tiered caching system by reducing internal traffic being the edge server and the forward server.