Application integration testing

Abstract

Application testing is disclosed. A definition of a test to be performed on a subject application is received in a generic form not specific to the subject application. The test is performed by exchanging data with the subject application, as required to perform the test, using a test connector application associated with the subject application to do at least one of (1) convert an input data to be supplied to the subject application during the test from a generic data format not specific to the subject application into an application-specific data format associated with the subject application, if the application-specific data format is different than the generic data format and (2) normalize an output data received from the subject application in the application-specific data format into the generic data format not specific to the subject application, if the application-specific data format is different than the generic data format.

1 . A method of testing an application, comprising:
 receiving a definition of a test to be performed on a subject application; and   performing the test;   wherein performing the test comprises exchanging data with the subject application, as required to perform the test, using a test connector application associated with the subject application to do at least one of (1) convert an input data to be supplied to the subject application during the test from a generic data format not specific to the subject application into an application-specific data format associated with the subject application, if the application-specific data format is different than the generic data format and (2) normalize an output data received from the subject application in the application-specific data format into the generic data format not specific to the subject application, if the application-specific data format is different than the generic data format.   
     
     
         2 . A method as recited in  claim 1 , wherein the subject application comprises two or more integrated subject applications and performing the test comprises using a separate test connector application specific to each of at least a subset of said two or more integrated subject applications to do with respect to the integrated subject application with which it is associated at least one of (1) convert an input data to be supplied to the subject application during the test from a generic data format not specific to the subject application into an application-specific data format associated with the subject application, if the application-specific data format is different than the generic data format and (2) normalize a test runtime output data received from the subject application in the application-specific data format into the generic data format not specific to the subject application, if the application-specific data format is different than the generic data format. 
     
     
         3 . A method as recited in  claim 1 , wherein performing the test comprises correlating a test runtime output data received during the test with a corresponding baseline data associated with the test. 
     
     
         4 . A method as recited in  claim 1 , wherein performing the test comprises comparing a test runtime output data received during the test to a corresponding baseline data associated with the test. 
     
     
         5 . A method as recited in  claim 1 , wherein performing the test comprises comparing a test runtime output data received during the test to a corresponding baseline data associated with the test and taking responsive action if it is determined the test runtime output data is different than the corresponding baseline data. 
     
     
         6 . A method as recited in  claim 1 , further comprising generating a set of baseline data associated with the test by running a baseline iteration of the test using a predefined test case and capturing one or more elements of baseline data generated during the baseline iteration of the test. 
     
     
         7 . A method as recited in  claim 1 , further comprising deriving a validation rule associated with the test by performing a baseline iteration of the test using a predefined test case, capturing one or more elements of baseline data generated during the baseline iteration of the test, and deriving the validation rule based at least in part on at least a subset of the one or more elements of captured baseline data. 
     
     
         8 . A method as recited in  claim 1 , wherein the test connector application is one of a plurality of test connector applications, each specific to a corresponding subject application for which testing is supported. 
     
     
         9 . A method as recited in  claim 1 , wherein the test connector application is selected from a plurality of connector applications, each specific to a corresponding subject application for which testing is supported. 
     
     
         10 . A method as recited in  claim 1 , wherein the test comprises a first test, the subject application comprises a first subject application, the definition comprises a first definition, and the method further comprises:
 receiving in the generic form a second definition of a second test to be performed on a second subject application; and   receiving an update rule defining a dynamic dependency between a dependent input to be supplied to the second application as part of the second test and a corresponding output generated by the first subject application when the first test is run and requiring that the value of the dependent input be updated based at least in part on the value of the corresponding output at a time subsequent to the corresponding output being generated during the running of the first test and prior to the dependent input being provided to the second subject application during the running of the second test.   
     
     
         11 . A method as recited in  claim 10 , wherein performing the test comprises doing in the order listed:
 performing the first test;   updating the dependent input based on the value of the corresponding output as generated during the running of the first test; and   performing the second test   
     
     
         12 . A method as recited in  claim 1 , wherein performing the test comprises automatically setting a testing environment associated with the test to an initial condition associated with the test. 
     
     
         13 . A method as recited in  claim 1 , wherein performing the test comprises automatically cleaning up a testing environment associated with the test to restore the testing environment at least in part to a condition in which it was prior to the test being run. 
     
     
         14 . A method as recited in  claim 1 , wherein the application-specific data format comprises a complex data format. 
     
     
         15 . A method as recited in  claim 1 , wherein the application-specific data format comprises a custom complex data format. 
     
     
         16 . A method as recited in  claim 1 , further comprising:
 receiving, in the event the subject application fails one or more aspects of the test, one or more failure notifications, each associated with a failure event; and   associating related ones of said one or more failure notifications, if any, together and assigning an error identification to each associated group of failure notifications and to each individual failure notification that is not associated with any other failure notification.   
     
     
         17 . A method as recited in  claim 16 , wherein the test involves a plurality of subject applications, each of which having an application-specific test connector application associated with it, and further comprising associating each error identification that is specific to a particular subject application with a subject application associated with the failure notification(s) associated with the error identification. 
     
     
         18 . A method as recited in  claim 17 , wherein associating each error identification with a subject application comprises associating each error identification with an application-specific test connector application associated with the failure notification(s) associated with the error identification. 
     
     
         19 . A system for testing an application, comprising:
 a processor configured to:
 receive a definition of a test to be performed on a subject application; and 
 perform the test; and 
   a memory configured to store the definition;   wherein performing the test comprises exchanging data with the subject application, as required to perform the test, using a connector application associated with the subject application to do at least one of (1) convert an input data to be supplied to the subject application during the test from a generic data format not specific to the subject application into an application-specific data format associated with the subject application, if the application-specific data format is different than the generic data format and (2) normalize an output data received from the subject application in the application-specific data format into the generic data format not specific to the subject application, if the application-specific data format is different than the generic data format.   
     
     
         20 . A computer program product for testing an application, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:
 receiving a definition of a test to be performed on a subject application; and   performing the test;   wherein performing the test comprises exchanging data with the subject application, as required to perform the test, using a test connector application associated with the subject application to do at least one of (1) convert an input data to be supplied to the subject application during the test from a generic data format not specific to the subject application into an application-specific data format associated with the subject application, if the application-specific data format is different than the generic data format and (2) normalize an output data received from the subject application in the application-specific data format into the generic data format not specific to the subject application, if the application-specific data format is different than the generic data format.