System for automatic assessment of student learning

Abstract

A system for using a smartpen as a tool for automatically assessing student learning that is simple and accurate. The system has a computer with at least one processor, a memory, a storage, and communications through a protocol, one or more than one smartpen in communication with the computer and a plurality of software modules having non-transitory instructions executable on the processor for at least a characterization module, for characterizing spatial and temporal organization of a student's work, and a machine learning module utilizing the spatial and temporal organization to assess student learning.

What is claimed is: 
     
       1. A computer assisted method for using smartpen outputs from a student for automatically assessing the student's learning performance, comprising:
 monitoring a smartpen output associated with the student to create a raw activity history for that student; 
 characterizing spatial and temporal organization of the raw activity history for that student to thereby create a characterized solution history for that student; and 
 analyzing the characterized solution history of the student to thereby assess the student's learning progress, 
 wherein the student's smartpen output is time stamped and the characterized solution history associated with the student includes identification of any smartpen output that was subsequently modified or erased by that student. 
 
     
     
       2. The method of  claim 1 , wherein the characterized solution history associated with the student comprises a plurality of discrete activities including drawing diagrams, revising diagrams, writing equations, revising equations, work on other problems, and no activity. 
     
     
       3. The method of  claim 2 , wherein the characterized solution history of the student includes
 a) recognized shapes of common symbols and letters; 
 b) classification of pen strokes; and 
 c) grouping of pen strokes based on the distance and elapsed time between the strokes. 
 
     
     
       4. The method of  claim 1 , wherein the temporal organization of the raw activity history is characterized at least in part by one or more of diagrams, free body diagram effort, equation effort, entropy, complexity, and breaks. 
     
     
       5. The method of  claim 1 , wherein the spatial organization of the raw activity history is characterized at least in part by out-of-order strokes and earlier-neighbor strokes. 
     
     
       6. The method of  claim 1 , wherein the characterized solution history includes groupings of solution elements into spatial clusters. 
     
     
       7. The method of  claim 6 , wherein the spatial and temporal organizations of the characterized solution history are computed and described by solution features computed from the spatial clusters. 
     
     
       8. The method of  claim 7 , wherein the solution features computed from the spatial clusters include the number of diagrams, free body diagram clusters, free body diagram revisits, free body diagram revisit strokes, number of equation clusters, equation area fractions, equation revisits, equation revisit strokes, or any combination thereof. 
     
     
       9. The method of  claim 7 , wherein the solution features computed from the spatial clusters include free body diagram strokes crossed-out, equation strokes crossed-out, big-cross-outs, typo-cross-outs, and ps-cross-outs. 
     
     
       10. The method of  claim 1 , wherein the characterized solution history includes time spent writing by the student, median free body diagram stroke length, median equation stroke length, median cross-out stroke length, number of free body diagram strokes, number of equation strokes, and number of cross-out strokes. 
     
     
       11. The method of  claim 1 , wherein a probability of correctness of the student's solution to a problem is calculated from features in the characterized solution history. 
     
     
       12. The method of  claim 1 , wherein a probability of student performance in a course is calculated from features in the characterized solution history. 
     
     
       13. The method of  claim 1 , wherein a student at risk of poor performance in a course is detected based on their characterized solution history. 
     
     
       14. The method of  claim 1 , further comprising the steps of:
 a) recognizing letters, mathematical symbols, arrows, and boxes; 
 b) classifying single strokes into one of at least three semantic classes; 
 c) grouping classified strokes; and 
 d) correcting errors within each group. 
 
     
     
       15. The method of  claim 1 , further comprising the step of semantically classifying strokes using an automatic character recognition process selected from the group consisting of an image-based recognizer, a domain-specific recognizer, and a single-character recognizer. 
     
     
       16. The method of  claim 1 , wherein a student at risk of poor performance in a course is detected based on comparing the student's characterized solution history to one or more other student's characterized solution histories.