Systems and methods for predicting performance

Abstract

The present disclosure relates to system and methods for predicting performance caused by software code changes. For this purpose, an augmented machine learning model predicts a latency of software module with updated code executed in a production environment. In some aspects, the latency is predicted based on a change of deviation that is determined by comparing the latency of the software module with updated code and the latency of the software module without updated code, whereas the software modules are executed in environments different from the production environment.

What is claimed is: 
     
       1. A system for predicting performance caused by software code changes comprising:
 a non-transitory memory storing instructions; and 
 one or more processors coupled to the non-transitory memory and configured to read the instructions from the non-transitory memory to cause the system to perform operations comprising: 
 obtaining for each of a first environment, a second environment and a stage environment a parameter set indicative of a latency of one or more process flows performed by a plurality of software modules executed on each of the first, second and stage environments, wherein at least one software module on the stage environment comprises updated code compared to the respective module on the first and second environment; 
 determining a deviation of the latency of the one or more process flows between the stage environment and the second environment; and 
 predicting based on the determined deviation, by an augmented machine learning model, a latency for the one or more process flows performed by the plurality of software modules executed on the first environment. 
 
     
     
       2. The system of  claim 1 , wherein the parameter set comprises an average time for performing each of the one or more process flows performed by the plurality of software modules. 
     
     
       3. The system of  claim 1 , wherein obtaining a parameter set comprises at least one of:
 receiving from each of the first, the second and the stage environments a time period corresponding to the latency of the one or more process flows; 
 receiving from each of the first, the second and the stage environments two or more time stamps corresponding to start and end times of the one or more process flows; and 
 storing the parameter set indicative of the latency in a storage system connected to the non-transitory memory and the one or more processors. 
 
     
     
       4. The system of  claim 1 , wherein the parameter set further comprises at least one of:
 an identifier of the process flow; 
 an identifier of the environment; 
 status of one or more hardware components associated with the environment; 
 performance parameters of one or more hardware components associated with the environment; 
 location or address of an initiator; or 
 location of any participant of the one or more process flows. 
 
     
     
       5. The system of  claim 1 , wherein the one or more process flows comprise two or more separable steps, each separable step associated with a step latency, and wherein obtaining the parameter set indicative of a latency comprises:
 obtaining the step latency for each of the two or more separable steps. 
 
     
     
       6. The system of  claim 1 , wherein predicting by an augmented machine learning model comprises:
 determining based on the parameter set associated with the latency for which a first deviation has been determined, by the augmented machine learning model, whether the first deviation has exceeded a pre-defined plausibility threshold; and 
 
       in response to the determination that the first deviation has exceeded the pre-defined plausibility threshold,
 discarding the first deviation and/or the parameter set associated with the latency for which a first deviation has been determined. 
 
     
     
       7. The system of  claim 6 , wherein the plausibility threshold is based on a predefined score. 
     
     
       8. The system of  claim 1 , wherein the augmented machine learning model utilizes a decision tree learning as its prediction model. 
     
     
       9. The system of  claim 1 , wherein predicting a latency comprises:
 predicting the latency in response to one or more parameter sets of the one or more process flows performed by the plurality of software modules executed on the first environment. 
 
     
     
       10. The system of  claim 9 , further comprising:
 determining whether a predicted latency exceeds a pre-defined threshold; and 
 
       in response to the determination that the predicted latency has exceeded the pre-defined threshold at least one of:
 flagging the respective one or more process flows; and 
 flagging the updated code. 
 
     
     
       11. The system of  claim 1 , wherein determining a deviation of the latency or predicting the latency comprises:
 classifying the latencies of the one or more process flows based on the parameter set associated with the respective process flow; and 
 generating, for each of the one or more process flows, an average latency from latencies sharing the same or similar class. 
 
     
     
       12. The system of  claim 1 , wherein the one or more process flows relate to at least one of:
 an online financial transaction; 
 an electronic transfer of assets; and 
 a transfer of cryptocurrency. 
 
     
     
       13. A computer-implemented method comprising:
 obtaining for each of a first environment, a second environment and a stage environment a parameter set indicative of a latency of one or more process flows performed by a plurality of software modules executed on each of the first, second and stage environment, wherein at least one software module on the stage environment comprises updated code compared to the respective module on the first and second environment; 
 determining a deviation of the latency of the one or more process flows between the stage environment and the second environment; and 
 predicting based on the determined deviation, by an augmented machine learning model, a latency for the one or more process flows performed by the plurality of software modules executed on the first environment. 
 
     
     
       14. The computer-implemented method of  claim 13 , wherein obtaining a parameter set comprises at least one of:
 receiving from each of the first, the second and the stage environment a time period corresponding to the latency of the one or more process flows; 
 receiving from each of the first, the second and the stage environments two or more time stamps corresponding to start and end times of the one or more process flows; and 
 storing the parameter set indicative of the latency in a storage system. 
 
     
     
       15. The computer-implemented method of  claim 13 , wherein the parameter set further comprises at least one of:
 an identifier of the process flow; 
 an identifier of the environment; 
 status of one or more hardware components associated with the environment; 
 performance parameters of one or more hardware components associated with the environment; 
 location or address of an initiator; or 
 location of any participant of the one or more process flows. 
 
     
     
       16. The computer-implemented method of  claim 13 , wherein predicting by an augmented machine learning model comprises:
 determining based on the parameter set associated with the latency for which a first deviation has been determined, by the augmented machine learning model whether the first deviation has exceeded a pre-defined plausibility threshold; and 
 
       in response to the determination that the first deviation has exceeded the pre-defined plausibility threshold,
 discarding the first deviation and/or the parameter set associated with the latency for which the first deviation has been determined. 
 
     
     
       17. The computer-implemented method of  claim 13 , wherein the augmented machine learning model utilizes a decision tree learning model as its prediction model. 
     
     
       18. The computer-implemented method of  claim 13 , wherein predicting a latency comprises:
 predicting the latency in response to one or more parameter sets of the one or more process flows performed by the plurality of software modules executed on the first environment. 
 
     
     
       19. The computer-implemented method of  claim 13 , wherein determining a deviation of the latency or predicting the latency comprises:
 classifying the latencies of the one or more process flows based on the parameter set associated with the respective process flow; and 
 generating, for each of the one or more process flows, an average latency from latencies sharing the same or similar class. 
 
     
     
       20. A non-transitory machine readable medium having stored thereon machine readable instructions executable to cause a machine to perform operations comprising:
 obtaining for each of a first environment, a second environment and a stage environment a parameter set indicative of a latency of one or more process flows performed by a plurality of software modules executed on each of the first, second and stage environments, wherein at least one software module on the stage environment comprises updated code compared to the respective module on the first and second environments; 
 determining a deviation of the latency of the one or more process flows between the stage environment and the second environment; and 
 predicting based on the determined deviation, by an augmented machine learning model, a latency for the one or more process flows performed by the plurality of software modules executed on the first environment.