Test Case Generation Through Reinforcement Learning With Static Code Quality Rewards

Abstract

A deep learning model is trained to learn to generate a better-quality unit test case for a focal method through reinforcement learning using a reward score that considers static code quality properties of a best coding standard. The static code quality properties include an assertion in the predicted unit test case, an invocation of the focal method in the predicted unit test case, and a descriptive name for the predicted unit test case. A reward model is trained to compute a reward score for a model-predicted unit test case based on the static code quality properties. The reward score is used in a proximal policy optimization method to produce a policy loss that updates the parameters of the deep learning model towards generating a better-quality unit test case.

What is claimed: 
     
         1 . A system comprising:
 a processor; and   a memory that stores a program configured to be executed by the processor, the program includes instructions to perform actions that:   obtain a reward model trained to generate a reward score for a model-generated unit test case for a focal method, wherein the reward score is based on a unit test case having one or more static code quality properties, wherein the one or more static code quality properties comprise an assertion, an invocation of the focal method, a name for the unit test case that describes the test case, and/or the existence of a comment that describes the test case;   obtain a deep learning model trained to learn to generate a unit test case, wherein parameters of the deep learning model are learned through a cross-entropy loss; and   tune the parameters of the deep learning model to learn to generate a unit test case having the one or more static code quality properties through reinforcement learning, wherein the reinforcement learning generates a policy loss that is based on a reward that comprises a reward score from the reward model for a unit test case generated by the tuned deep learning model for a given tuning sample, wherein the policy loss is backpropagated through the tuned deep learning model.   
     
     
         2 . The system of  claim 1 , wherein the program includes instructions to perform actions that:
 deploy the tuned deep learning model in a source code development system to generate a target unit test case for a given target focal method.   
     
     
         3 . The system of  claim 1 , wherein the one or more static code quality properties comprises syntax correctness of the unit test case generated by the tuned first deep learning model. 
     
     
         4 . The system of  claim 1 , wherein the one or more static code quality properties comprises an absence of a duplicate assertion in the unit test case generated by the tuned first deep learning model. 
     
     
         5 . The system of  claim 1 , wherein the one or more static code quality properties comprise an absence of conditional logic in the unit test case generated by the tuned first deep learning model. 
     
     
         6 . The system of  claim 1 , wherein the one or more static code quality properties comprise an absence of exception handling code, a print statement, and/or an empty test in the unit test case generated by the tuned first deep learning model. 
     
     
         7 . The system of  claim 1 , wherein the reward model is a neural transformer model with attention. 
     
     
         8 . The system of  claim 1 , wherein the deep learning model is a neural transformer model with attention. 
     
     
         9 . A computer-implemented method, comprising:
 obtaining a reward model trained to generate a reward score for a model-generated unit test case for a focal method, wherein the quality score is based on a unit test case having a plurality of static code quality properties, wherein the plurality of static code quality properties comprises an assertion, an invocation of the focal method, a name for the unit test case that describes the focal method, and/or the existence of a comment that describes the test case;   fine-tuning a large language model to learn to generate a unit test case for a given focal method and context of the focal method, wherein the large language model was pre-trained to generate source code;   tuning the large language model to learn to generate a unit test case having the plurality of static code quality properties, wherein the tuning uses a policy gradient method to determine a policy loss that is backpropagated to layers of the tuned large language model, wherein the policy loss is based on a reward having a reward score generated by the reward model for a unit test case generated by the tuned large language model for a given tuning sample; and   deploying the tuned large language model in a software development environment to predict a target unit test case for a target focal method.   
     
     
         10 . The computer-implemented method of  claim 9 , wherein the plurality of static code properties further comprises one or more anti-pattern properties, wherein an anti-pattern property comprises existence of a duplicate assertion in the unit test case generated by the tuned large language model. 
     
     
         11 . The computer-implemented method of  claim 10 , wherein the one or more anti-pattern properties further comprise existence of conditional logic or exception handling code in the unit test case generated by the tuned large language model. 
     
     
         12 . The computer-implemented method of  claim 11 , wherein the one or more anti-pattern properties further comprise existence of a print statement in the unit test case generated by the tuned large language model. 
     
     
         13 . The computer-implemented method of  claim 12 , wherein the one or more anti-pattern properties further comprise existence of an empty test in the unit test case generated by the tuned large language model. 
     
     
         14 . The computer-implemented method of  claim 10 , further comprising:
 generating the reward score as a combination of positive values for existence of the assertion, invocation of the focal method, existence of the name for the unit test case that describes the test case, and existence of a comment that describes the test case or the absence of anti-patterns, and a negative value for each syntactically incorrect unit test case generated by the tuned first large language model.   
     
     
         15 . The computer-implemented method of  claim 9 , wherein the reward model is a neural transformer model with attention. 
     
     
         16 . A hardware storage device having stored thereon computer executable instructions that are structured to be executable by a processor of a computing device to thereby cause the computing device to perform actions for:
 accessing a reward model trained to generate a reward score for a model-generated unit test case for a focal method, wherein the reward score is based on a unit test case having one or more of a plurality of static code quality properties, wherein the plurality of static code quality properties comprises an assertion, an invocation of the focal method, a name for the unit test case that describes the test case, and existence of a comment that describes the test case;   accessing a neural-based model trained to generate a unit test case for a focal method, wherein the neural-based model comprises a plurality of parameters generated via a cross-entropy loss; and   tuning the neural-based model to learn to predict a unit test case for a target focal method having one or more of the plurality of static code quality properties, wherein the tuning of the neural-based model comprises:
 iteratively sampling a predicted unit test case generated by the neural-based model and a predicted unit test case generated by the tuned neural-based model for a tuning sample; 
 computing, by the reward model, a reward score for the predicted unit test case generated by the tuned neural-based model; 
 computing a reward-based loss for the predicted unit test case generated by the neural-based model and the predicted unit test case generated by the tuned neural-based model; 
 augmenting the reward-based loss with the reward score; and 
 updating parameters of the tuned neural-based model using a policy loss that is based on the augmented reward-based loss. 
   
     
     
         17 . The hardware device of  claim 16 , wherein the plurality of static code quality properties further comprises a plurality of anti-patterns, wherein the plurality of anti-patterns comprises existence of a duplicate assertion, existence of conditional logic or exception handling code, existence of print statement, and/or existence of an empty test in the unit test case generated by the tuned neural-based model. 
     
     
         18 . The hardware device of  claim 16 , wherein the reward model is trained to generate the reward score for a select one of the static code quality properties. 
     
     
         19 . The hardware device of  claim 16 , wherein the neural-based model comprises a neural transformer model with attention. 
     
     
         20 . The hardware device of  claim 16 , wherein the reward model comprises a neural transformer model with attention.