System and method for conducting multi-layer user selectable electronic testing

Abstract

A multi-layer user-selectable electronic testing (MUSET) system provides a cascaded set of alternative testing formats for test-takers to select a testing format that best accommodates their level of ability. Test-takers can answer fill-in-the-blank (FITB) items on a computer or other input device. If the test-taker is less confident of their understanding, they can select multiple-choice (MC) or true/false (T/F) testing formats. The MUSET system measures, tracks, and stores the amount of time it takes to answer test items, to switch testing formats, and to change answers. Test-takers indicate a confidence level that they have in the correctness of their answer. The MUSET system determines confidence characteristics, latency characteristics, and hesitancy characteristics of the test-taker and gathers additional parameters to build a performance profile of the test-taker's skills/traits/abilities. The performance profile is analyzed to guide and inform evaluators regarding individual performance, trends over time, differences between test-taker subsets, and analyses of test items.

The claimed invention is: 
     
         1 . A computer-implemented multi-layer user-selectable electronic testing method of building an electronic test-taker performance profile comprising executing on a computer processor the steps of:
 receiving a results file from a test session database, the results file appended to include:
 an electronic test item format type selected by the test-taker; 
 an amount of time taken by the test-taker to select the electronic test item format type; 
 an assigned test item format value multiplier that is based upon a predetermined difficulty of the electronic test item format type selected by the test-taker; 
 an assigned test item difficulty value that is based upon perceived predetermined difficulty of subject matter of an electronic test item; 
 an electronic test item answer to the electronic test item provided by the test-taker; 
 an amount of time taken by the test-taker to provide the electronic test item answer; 
 a confidence level selected by the test-taker indicating their level of confidence in the correctness of their electronic test item answer; 
 an amount of time taken by the test-taker to select their confidence level; 
 a confidence weighting value from a database of global parameters; 
 a latency score based on the amount of time taken by the test-taker to select the electronic test item format type, the amount of time taken by the test-taker to provide the answer to the electronic test item, and the amount of time taken by the test-taker to provide the confidence level; 
 a hesitancy score based on a number of changes made by the test-taker to select the electronic test item format type, a number of changes made by the test-taker to provide the answer to the electronic test item, and a number of changes made by the test-taker to provide the confidence level; 
 a computed test score based on a correct electronic test item answers, the assigned test item format value multiplier, the test item difficulty value, and the confidence weighting value; and 
   building the electronic test-taker performance profile based on the computed test score, confidence level, hesitancy score, and latency score of the results file.   
     
     
         2 . The computer-implemented multi-layer user-selectable electronic testing method of building an electronic test-taker performance profile of  claim 1  further comprising:
 receiving a testing session data file including the electronic test item answer from a test session database; 
 receiving a test item parameter file including a test specifications file from a database of global parameters; 
 synchronizing the testing session data file and the test specifications file for a test item; and 
 creating the results file, wherein the results file is a matching results file for the test item based upon structures of the synchronized testing session data file and the test specifications file. 
 
     
     
         3 . The computer-implemented multi-layer user-selectable electronic testing method of building an electronic test-taker performance profile of  claim 1 , wherein the assigned test item format value multiplier is appended to the test session results file from a test item parameter file (F 1 ) upon selection of the test item format by the test-taker. 
     
     
         4 . The computer-implemented multi-layer user-selectable electronic testing method of building an electronic test-taker performance profile of  claim 1 , wherein the test item difficulty value is appended to the test session results file from a test item parameter file (F 1 ) upon selection of the test item format by the test-taker. 
     
     
         5 . The multi-layer user-selectable electronic testing method of  claim 1  further comprising:
 computing a confidence ratio determined by the average percent confidence for correctly answered test items divided by the average percent confidence for incorrectly answered test items. 
 
     
     
         6 . The multi-layer user-selectable electronic testing method of  claim 5  further comprising:
 computing separate confidence ratios for test items presented to the test taker in each test item format. 
 
     
     
         7 . The multi-layer user-selectable electronic testing method of  claim 1  further comprising:
 computing a latency ratio determined by the average amount of time it takes for the test taker to answer incorrectly answered test items divided by the average amount of time it takes for the test taker to answer correctly answered test items. 
 
     
     
         8 . The multi-layer user-selectable electronic testing method of  claim 7  further comprising:
 computing separate latency ratios for test items presented to the test taker in each test item format. 
 
     
     
         9 . The multi-layer user-selectable electronic testing method of  claim 1 , wherein the electronic test item format type is a multiple-choice test item format, and the hesitancy score is determined by an average number of times the test-taker changes an answer selected on a multiple-choice test item. 
     
     
         10 . The multi-layer user-selectable electronic testing method of  claim 1  further comprising:
 computing an item difficulty ratio determined by the number of test items answered using an initial test item format divided by the number of test items answered using an alternative test item format. 
 
     
     
         11 . The multi-layer user-selectable electronic testing method of  claim 10  further comprising:
 computing separate item difficulty ratios for test items presented to the test taker in each test item format. 
 
     
     
         12 . The multi-layer user-selectable electronic testing method of  claim 10  further comprising:
 computing a contrasted item difficulty ratio for test items presented to the test taker in an initial test item format determined by a number of correct answers to test items using the initial test item format divided by a number of incorrect answers to test items using the initial test item format. 
 
     
     
         13 . The multi-layer user-selectable electronic testing method of  claim 10  further comprising:
 computing a contrasted item difficulty ratio for test items presented to the test taker in the alternative test item format determined by a number of correct answers to test items using the alternative test item format divided by the number of incorrect answers to test items using the alternative test item format. 
 
     
     
         14 . The multi-layer user-selectable electronic testing method of  claim 1  further comprising:
 comparing a test-taker's number of correct answers to a predetermined range of number of correct answers; 
 categorizing the test-taker as highly qualified when the test-taker's number of correct answers is greater than an upper limit of the predetermined range of number of correct answers; 
 categorizing the test-taker as qualified when the test taker's number of correct answers is less than or equal to the upper limit of the predetermined range of number of correct answers and greater than or equal to a lower limit of the predetermined range of number of correct answers; and 
 categorizing the test-taker as not qualified when the test taker's number of correct answers is fewer than the lower limit of the predetermined range of number of correct answers. 
 
     
     
         15 . The multi-layer user-selectable electronic testing method of  claim 5  further comprising:
 comparing a test-taker's confidence ratio to a predetermined range of confidence ratios; 
 categorizing the test-taker as keenly aware when the test-taker's confidence ratio is greater than an upper limit of the predetermined range of confidence ratios; 
 categorizing the test-taker as aware when the test taker's confidence ratio is less than or equal to the upper limit of the predetermined range of confidence ratios and greater than or equal to a lower limit of the predetermined range of confidence ratios; and 
 categorizing the test-taker as unaware when the test taker's confidence ratio is less than the lower limit of the predetermined range of confidence ratios. 
 
     
     
         16 . The multi-layer user-selectable electronic testing method of  claim 9  further comprising:
 comparing a test-taker's hesitancy score to a predetermined range of hesitancy scores; 
 categorizing the test-taker as bold when the test-taker's hesitancy score is lower than a lower limit of the predetermined range of hesitancy scores; 
 categorizing the test-taker as average when the test taker's hesitancy score is less than or equal to the upper limit of the predetermined range of hesitancy score and greater than or equal to a lower limit of the predetermined range of hesitancy scores; and 
 categorizing the test-taker as hesitant when the test taker's hesitancy score is greater than the upper limit of the predetermined range of hesitancy scores. 
 
     
     
         17 . The multi-layer user-selectable electronic testing method of  claim 1  further comprising:
 appending the test session results file with a multi-dimensional test-taker performance profile analysis grid based on a predetermined schema to categorize performance of the test taker. 
 
     
     
         18 . The multi-layer user-selectable electronic testing method of  claim 17 , wherein the predetermined schema is chosen by a testing entity. 
     
     
         19 . The multi-layer user-selectable electronic testing method of  claim 17  further comprising:
 computing a confidence ratio determined by the average percent confidence for correctly answered test items divided by the average percent confidence for incorrectly answered test items, and 
 wherein the multi-dimensional test-taker performance profile analysis grid includes the computed test score, the latency score, and the confidence ratio as the dimensions of the multi-dimensional performance profile analysis grid. 
 
     
     
         20 . The multi-layer user-selectable electronic testing method of  claim 17  further comprising:
 computing a confidence ratio determined by the average percent confidence for correctly answered test items divided by the average percent confidence for incorrectly answered test items, and 
 wherein the multi-dimensional test-taker performance profile analysis grid includes at least two of the computed test score, the latency score, the confidence ratio, and the hesitancy score as the dimensions of the performance profile analysis grid.