Maze-driven self-diagnostics using reinforcement learning

Abstract

Systems and methods are provided for automatedly troubleshooting a computing application (e.g., a cloud-based computing application). An application domain of the computing application is modeled as a two-dimensional array of cells, a first dimension of the array representing components or microservices of the application domain, and a second dimension of the array representing states of the components or microservices, the array including paths between pairs of cells in the array. A troubleshooting goal is defined as a target state of the application domain, the target state corresponding to a target cell in the array. An initial state of the application domain is also provided, the initial state corresponding to an initial cell in the array. A reinforcement-learning-trained machine-learning algorithm can determine a solution path in the array between the initial cell and the target cell. Divergence between a failure case and a solution path indicates a probable failure cause.

1 - 20 . (canceled) 
     
     
         21 . A method of troubleshooting a computing application in a distributed computing environment, the method comprising:
 generating an array of cells, wherein each cell of the array of cells represents a state of a component;   defining a troubleshooting goal corresponding to a target cell in the array;   determining a solution path in the array between an initial cell and the target cell;   determining a failed point along the solution path by comparing failure data with the solution path; and   re-writing or replacing a failed component.   
     
     
         22 . The method of  claim 21 , wherein determining the solution path in the array between the initial cell and the target cell includes using a reinforcement machine learning algorithm. 
     
     
         23 . The method of  claim 21 , further comprising:
 generating a diagnostic message indicating the failure point as a failure reason or a solution suggestion responsive to the troubleshooting goal.   
     
     
         24 . The method of  claim 21 , further comprising:
 expanding the array in at least one of a first dimension or a second dimension by one or both of adding one or more components to the array or by adding to the array one or more states of the components in the array.   
     
     
         25 . The method of  claim 21 , wherein generating the array comprises extracting information from a log file associated with the component of the computing application. 
     
     
         26 . The method of  claim 21 , further comprising:
 providing an initial state corresponding to an initial cell in the array, wherein the initial state corresponds to a starting state of the computing application.   
     
     
         27 . The method of  claim 21 , wherein the target cell corresponds to a target state of the computing application. 
     
     
         28 . A self-diagnostics system for troubleshooting a computing application in a distributed computing environment, the self-diagnostics system comprising one or more computer processors coupled to a non-transitory memory storing instructions configured to, when executed by the one or more computer processors:
 generate an array of cells, wherein each cell of the array of cells represents a state of a component;   define a troubleshooting goal corresponding to a target cell in the array;   determine a solution path in the array between an initial cell and the target cell;   determine a failed point along the solution path by comparing failure data with the solution path; and   re-write or replace a failed component.   
     
     
         29 . The self-diagnostics system of  claim 28 , wherein the instructions are further configured to determine the solution path in the array between the initial cell and the target cell using a reinforcement machine learning algorithm. 
     
     
         30 . The self-diagnostics system of  claim 28 , wherein the instructions are further configured to generate a diagnostic message indicating the failure point as a failure reason or a solution suggestion responsive to the troubleshooting goal. 
     
     
         31 . The self-diagnostics system of  claim 28 , wherein the instructions are further configured to:
 expand the array in at east one of a first dimension or a second dimension by one or both of adding one or more components to the array or by adding to the array one or more states of the components in the array.   
     
     
         32 . The self-diagnostics system of  claim 28 , wherein generating the array comprises extracting information from a log file associated with the component of the computing application. 
     
     
         33 . The self-diagnostics system of  claim 28 , wherein the instructions are further configured to provide an initial state corresponding to an initial cell in the array, wherein the initial state corresponds to a starting state of the computing application. 
     
     
         34 . The self-diagnostics system of  claim 28 , wherein the target cell corresponds to a target state of the computing application. 
     
     
         35 . A non-transitory computer-readable medium storing instructions that, when executed by a processor, cause the processor to:
 generate an array of cells, wherein each cell of the array of cells represents a state of a component;   define a troubleshooting goal corresponding to a target cell in the array;   determine a solution path in the array between an initial cell and the target cell;   determine a failed point along the solution path by comparing failure data with the solution path; and   re-write or replace a failed component.   
     
     
         36 . The non-transitory computer-readable of  claim 35 , wherein the instructions are further configured to determine the solution path in the array between the initial cell and the target cell using a reinforcement machine learning algorithm. 
     
     
         37 . The non-transitory computer-readable of  claim 35 , wherein the instructions are further configured to generate a diagnostic message indicating the failure point as a failure reason or a solution suggestion responsive to the troubleshooting goal. 
     
     
         38 . The non-transitory computer-readable of  claim 35 , wherein the instructions are further configured to:
 expand the array in at east one of a first dimension or a second dimension by one or both of adding one or more components to the array or by adding to the array one or more states of the components in the array.   
     
     
         39 . The non-transitory computer-readable of  claim 35 , wherein generating the array comprises extracting information from a log file associated with the component of the computing application. 
     
     
         40 . The non-transitory computer-readable of  claim 35 , wherein the instructions are further configured to provide an initial state corresponding to an initial cell in the array, wherein the initial state corresponds to a starting state of the computing application.