Continuous deployment pipeline tests

Abstract

In some examples, a system provides a plurality of code changes of a program in a continuous deployment pipeline that performs tests on the plurality of code changes and deploys a code assembly including the plurality of code changes. The system ranks a plurality of tests based on a test attribute and a test context to provide a test rank for each of the plurality of tests, and selects, from among the plurality of tests using the test ranks, a subset of tests to include in a test set for the continuous deployment pipeline. The system executes the test set in the continuous deployment pipeline to test the plurality of code changes.

What is claimed is: 
     
         1 . A method comprising:
 providing, by a system comprising a processor, a plurality of code changes of a program in a continuous deployment pipeline that performs tests on the plurality of code changes and deploys a code assembly including the plurality of code changes;   ranking, by the system, a plurality of tests based on a test attribute and a test context to provide a test rank for each of the plurality of tests;   selecting, by the system from among the plurality of tests using the test ranks, a subset of tests to include in a test set for the continuous deployment pipeline; and   executing, by the system, the test set in the continuous deployment pipeline to test the plurality of code changes.   
     
     
         2 . The method of  claim 1 , wherein the selecting is according to a criterion relating to execution time of performing tests in the continuous deployment pipeline. 
     
     
         3 . The method of  claim 1 , wherein the selecting is according to a criterion relating to a cost of performing tests in the continuous deployment pipeline. 
     
     
         4 . The method of  claim 1 , further comprising keeping the plurality of code changes in the continuous deployment pipeline until the code assembly with the plurality of code changes passes the test set. 
     
     
         5 . The method of  claim 1 , further comprising receiving, in the continuous deployment pipeline, the plurality of code changes that match a specified criterion of the continuous deployment pipeline. 
     
     
         6 . The method of  claim 1 , wherein the continuous deployment pipeline is a first continuous deployment pipeline, the method further comprising:
 receiving, in a second continuous deployment pipeline, a further plurality of code changes that match a specified criterion of the second continuous deployment pipeline;   ranking a further plurality of tests based on a test attribute and a test context to provide a test rank for each of the further plurality of tests;   selecting, from among the further plurality of tests using the test ranks for the further plurality of tests, a subset of tests to include in a second test set for the second continuous deployment pipeline; and   executing the second test set in the second continuous deployment pipeline to test the further plurality of code changes.   
     
     
         7 . The method of  claim 1 , further comprising:
 collecting a test result from the executed test set, the test result including data corresponding to at least one of the test attribute and the test context.   
     
     
         8 . The method of  claim 1 , wherein the test attribute comprises data associated with the plurality of tests and previous executions of the plurality of tests. 
     
     
         9 . The method of  claim 1 , wherein the test context comprises the plurality of code changes. 
     
     
         10 . The method of  claim 1 , wherein the test attribute and the test context are each assigned a weighted value is determined based on the continuous deployment pipeline. 
     
     
         11 . A system to prioritize a plurality of tests in a continuous deployment pipeline, the system comprising:
 a processor; and   a non-transitory storage medium storing instructions executable on the processor to:
 receive a plurality of code changes of a program in a continuous deployment pipeline that performs tests on the plurality of code changes and deploys a code assembly including the plurality of code changes; 
 rank a plurality of tests based on a test attribute and a test context to provide a test rank for each of the plurality of tests; 
 select, from among the plurality of tests using the test ranks, a subset of tests to include in a test set for the continuous deployment pipeline; and 
 execute the test set in the continuous deployment pipeline to test the plurality of code changes. 
   
     
     
         12 . The system of  claim 11 , wherein the test attribute comprises data associated with the plurality of tests and previous executions of the plurality of tests. 
     
     
         13 . The system of  claim 11 , wherein the test context comprises the plurality of code changes. 
     
     
         14 . The system of  claim 11 , wherein the selecting is according to a criterion relating to execution time of performing tests in the continuous deployment pipeline. 
     
     
         15 . The system of  claim 11 , wherein the selecting is according to a criterion relating to a cost of performing tests in the continuous deployment pipeline. 
     
     
         16 . The system of  claim 11 , wherein the instructions are executable on the processor to keep the plurality of code changes in the continuous deployment pipeline until the code assembly with the plurality of code changes passes the test set. 
     
     
         17 . The system of  claim 11 , wherein the instructions are executable on the processor to receive, in the continuous deployment pipeline, the plurality of code changes that match a specified criterion of the continuous deployment pipeline. 
     
     
         18 . A non-transitory machine-readable storage medium storing instructions that upon execution cause a system to:
 receive a plurality of code changes of a program in a continuous deployment pipeline that performs tests on the plurality of code changes and deploys a code assembly including the plurality of code changes;   rank a plurality of tests based on a test attribute and a test context to provide a test rank for each of the plurality of tests;   select, from among the plurality of tests using the test ranks, a subset of tests to include in a test set for the continuous deployment pipeline; and   execute the test set in the continuous deployment pipeline to test the plurality of code changes.   
     
     
         19 . non-transitory machine-readable storage medium system of  claim 15 , wherein the instructions are executable on the processor to keep the plurality of code changes in the continuous deployment pipeline until the code assembly with the plurality of code changes passes the test set. 
     
     
         20 . The non-transitory machine-readable storage medium of  claim 15 , wherein the instructions are executable on the processor to receive, in the continuous deployment pipeline, the plurality of code changes that match a specified criterion of the continuous deployment pipeline.