Mitigating failure in request handling

Abstract

A network computer system operates to mitigate failures for a network service. The network computer system can generate a data path model for the network service, where the data path model identifies a probabilistic set of expectations with respect to the programs and program sequences which handle service requests for the network service. The data path models can be used to detect, analyze or mitigate service request failures of the network service.

What is claimed is: 
     
         1 . A network computer system comprising:
 one or more processors; and   a set of memory resources, the set of memory resources storing a set of instructions, wherein the one or more processors access the instructions to:   determine logging information for a collection of service requests associated with a network service, the logging information including trace routes that identify multiple programs which execute to fulfill each service request of the collection;   generate one or more data path models that identify multiple possible data paths using the logging information, the one or more data path models comprising sequence data identifying a relative order by which at least some of the multiple programs execute to fulfill service requests;   determine, based on the one or more data path models, a status of one or more service requests handled by one or more of the multiple programs;   provide, on a device of a network operator, an interface to visually indicate the status of the one or more service requests; and   enable a user to interact with the interface to initiate a programmatic action to remediate or mitigate against an error that is indicated by the status of at least one of the one or more service requests.   
     
     
         2 . The network computer system of  claim 1 , wherein the programmatic action includes signaling a program to restart. 
     
     
         3 . The network computer system of  claim 1 , wherein the programmatic actions includes isolating a set of programs which are a candidate source for the error. 
     
     
         4 . The network computer system of  claim 1 , wherein the operations further comprise determining, based on the one or more data path models and the status information, one of the multiple programs that are a program source of the indicated error. 
     
     
         5 . The network computer system of  claim 4 , wherein the one or more processors access the instructions to perform analysis, using the data path models, to determine a potential root cause of a failure that is indicated by the detected error. 
     
     
         6 . The network computer system of  claim 1 , wherein the logging information includes timing information that is indicative of timing between individual programs of the multiple programs, and wherein the sequence data is based at least in part on the timing information. 
     
     
         7 . The network computer system of  claim 1 , wherein the interface includes a nodal representation of a process implemented using the one or more of the multiple programs. 
     
     
         8 . The network computer system of  claim 1 , wherein the interface includes a graphic representation of a data path for a failed service request up to when the service request was handled. 
     
     
         9 . The network computer system of  claim 8 , wherein the graphic representation is generated from trace route information for the failed service request. 
     
     
         10 . The network computer system of  claim 8 , wherein the one or more processors access the instructions to perform failure analysis, using trace route information, the failure analysis identifying one or more candidate programs that are a source of the failed service request, and wherein at least one of the one or more candidate programs is visually indicated on the interface. 
     
     
         11 . A computer-implemented method comprising:
 determining logging information for a collection of service requests associated with a network service, the logging information including trace routes that identify multiple programs which execute to fulfill each service request of the collection;   generating one or more data path models that identify multiple possible data paths using the logging information, the one or more data path models comprising sequence data identifying a relative order by which at least some of the multiple programs execute to fulfill service requests;   determining, based on the one or more data path models, a status of one or more service requests handled by one or more of the multiple programs;   providing, on a device of a network operator, an interface to visually indicate the status of the one or more service requests; and   enabling a user to interact with the interface to initiate a programmatic action to remediate or mitigate against an error that is indicated by the status of at least one of the one or more service requests.   
     
     
         12 . The method of  claim 11 , wherein the programmatic action includes signaling a program to restart. 
     
     
         13 . The method of  claim 11 , wherein the programmatic actions includes isolating a set of programs which are a candidate source for the error. 
     
     
         14 . The method of  claim 11 , wherein the method further comprises:
 determining, based on the one or more data path models and the status information, one of the multiple programs that are a program source of the indicated error.   
     
     
         15 . The method of  claim 14 , wherein the method further comprises:
 performing analysis, using the data path models, to determine a potential root cause of a failure that is indicated by the detected error.   
     
     
         16 . The method of  claim 11 , wherein the logging information includes timing information that is indicative of timing between individual programs of the multiple programs, and wherein the sequence data is based at least in part on the timing information. 
     
     
         17 . The method of  claim 11 , wherein the interface includes a nodal representation of a process implemented using the one or more of the multiple programs. 
     
     
         18 . The method of  claim 11 , wherein the providing the interface includes generating, from trace route information of the failed service request, a graphic representation of a data path for a failed service request up to when the service request was handled. 
     
     
         19 . The method of  claim 18 , wherein the method further comprises:
 performing failure analysis, using trace route information, the failure analysis identifying one or more candidate programs that are a source of the failed service request; and   wherein at least one of the one or more candidate programs is visually indicated on the interface.   
     
     
         20 . A non-transitory computer-readable medium that stores instructions, which when executed by one or more processors of a computer system, cause the computer system to perform operations comprising:
 determining logging information for a collection of service requests associated with a network service, the logging information including trace routes that identify multiple programs which execute to fulfill each service request of the collection;   generating one or more data path models that identify multiple possible data paths using the logging information, the one or more data path models comprising sequence data identifying a relative order by which at least some of the multiple programs execute to fulfill service requests;   determining, based on the one or more data path models, a status of one or more service requests handled by one or more of the multiple programs;   providing, on a device of a network operator, an interface to visually indicate the status of the one or more service requests; and   enabling a user to interact with the interface to initiate a programmatic action to remediate or mitigate against an error that is indicated by the status of at least one of the one or more service requests.