Systems and methods for feedback and evaluation

Abstract

Systems and methods may provide feedback and evaluation on the progress of a learner toward learner-defined goals. Inactive raters may be identified and inactivity alerts may be generated and sent to corresponding users. Goal ratings with observation notes may be identified and observation alerts may be generated and sent to corresponding users. Goal ratings having a rating differential that exceeds a threshold may be identified and delta alerts may be generated and sent to corresponding users. Sustained progress toward goals may be identified and progress alerts may be generated and sent to corresponding users. Average goal ratings for each of a set of corresponding categories may be identified for defined norm groups. Charts may be generated and displayed that provide a comparison between average learner goal ratings over a time segment and the performance of a selected norm group in a corresponding category.

1 . A system comprising:
 a server device comprising:
 a processor configured to execute instructions stored on a non-transitory computer readable storage medium for: 
 scanning a plurality of goal ratings stored in a database, the plurality of goal ratings corresponding to evaluations of goals of one or more learners; 
 identifying that no goal ratings have been submitted by a first rater for a goal of a learner of the one or more learners for a period of time exceeding a predefined inactivity threshold value; 
 generating first, second, and third inactivity alerts; 
 electronically sending the first inactivity alert to a first client computer device associated with a first user account of the learner; 
 electronically sending the second inactivity alert to a second client computer device associated with a second user account of a reviewer; and 
 electronically sending the third inactivity alert to a third client computer device associated with a third user account of a coach. 
   
     
     
         2 . The system of  claim 1 , wherein the processor is further configured to execute instructions for:
 determining that an observation note has been submitted with a goal rating of the plurality of goal ratings, the goal rating corresponding to the goal of the learner;   generating first and second observation alerts;   electronically sending the first observation alert to the first client computer device; and   electronically sending the second observation alert to the third client computer device.   
     
     
         3 . The system of  claim 1 , wherein the processor is further configured to execute instructions for:
 identifying a first goal rating submitted by a second rater within a predetermined time period for the goal of the learner;   identifying a second goal rating submitted by the learner for the goal within the predetermined time period;   determining a rating differential by calculating a difference between the first goal rating and the second goal rating;   determining that the rating differential exceeds a predefined delta threshold value;   generating first and second delta alerts;   electronically sending the first delta alert to the first client computer device; and   electronically sending the second delta alert to the third client computer device.   
     
     
         4 . The system of  claim 1 , wherein the processor is further configured to execute instructions for:
 calculating an average goal rating value by averaging all goal ratings submitted by raters over a predefined progress assessment time period for the goal of the learner;   calculating a progress percentage corresponding to the average goal rating value divided by a target goal rating value set by the learner;   determining that the progress percentage exceeds a predefined progress threshold value;   generating first, second, and third progress alerts;   electronically sending the first progress alert to the first client computer device;   electronically sending the second progress alert to the second client computer device; and   electronically sending the third progress alert to the third client computer device.   
     
     
         5 . The system of  claim 1 , wherein the processor is further configured to execute instructions for:
 scanning the plurality of goal ratings to identify a set of goal ratings corresponding to an unprocessed predefined group of learners of a set of predefined groups of learners;   calculating a first mean and a first standard deviation of a first subset of goal ratings of the set of goal ratings, the first subset of goal ratings corresponding to a first predefined category;   calculating a second mean and a second standard deviation of a second subset of goal ratings of the set of goal ratings, the second subset of goal ratings corresponding to a second predefined category;   generating a first chart that is divided into a first plurality of sections, the first plurality of sections corresponding to first goal rating ranges that are bounded based on the first mean and the first standard deviation;   generating a second chart that is divided into a second plurality of sections, the second plurality of sections corresponding to second goal rating ranges that are bounded based on the second mean and the second standard deviation; and   causing the first chart and the second chart to be displayed at the first client computer device.   
     
     
         6 . A method comprising:
 scanning, by a processor of a server device, a plurality of goal ratings stored in a database at the server device, the plurality of goal ratings corresponding to evaluations of goals of one or more learners;   identifying, by the processor, that a predefined condition has been met based on the plurality of goal ratings; and   electronically sending, by the processor via at least one electronic communication network, at least one alert to at least one client computer device based on identifying that the predefined condition has been met.   
     
     
         7 . The method of  claim 6 , wherein identifying that the predefined condition has been met comprises identifying that no goal ratings have been submitted by a first rater for a goal of a learner of the one or more learners for a period of time exceeding a predefined inactivity threshold value, and wherein electronically sending the at least one alert comprises:
 generating first, second, and third inactivity alerts;   electronically sending the first inactivity alert to a first client computer device associated with a first user account of the learner;   electronically sending the second inactivity alert to a second client computer device associated with a second user account of a reviewer; and   electronically sending a third inactivity alert to a third client computer device associated with a third user account of a coach.   
     
     
         8 . The method of  claim 6 , wherein identifying that the predefined condition has been met comprises determining that an observation note has been submitted with a goal rating of the plurality of goal ratings, the goal rating corresponding to a goal of a learner of the one or more learners, and wherein electronically sending the at least one alert comprises:
 generating first and second observation alerts;   electronically sending the first observation alert to a first client computer device associated with a first user account of the learner; and   electronically sending the second observation alert to a second client computer device associated with a second user account of a coach.   
     
     
         9 . The method of  claim 6 , wherein identifying that the predefined condition has been met comprises:
 identifying a minimum goal rating of a first set of goal ratings submitted by a second rater within a predetermined time period for a goal of a learner;   identifying a maximum goal rating of a second set of goal ratings submitted by the learner for the goal within the predetermined time period;   determining a rating differential at least by calculating a difference between the maximum goal rating and the minimum goal rating; and   determining that the rating differential exceeds a predefined delta threshold value.   
     
     
         10 . The method of  claim 9 , wherein electronically sending at least one alert comprises:
 generating first and second delta alerts;   electronically sending the first delta alert to a first client computer device associated with a first user account of the learner; and   electronically sending the second delta alert to a second client computer device associated with a second user account of a coach.   
     
     
         11 . The method of  claim 6 , wherein identifying that the predefined condition has been met comprises:
 calculating an average goal rating value by averaging all goal ratings submitted by raters over a predefined progress assessment time period for a goal of a learner;   calculating a progress percentage corresponding to the average goal rating value divided by a target goal rating value set by the learner; and   determining that the progress percentage exceeds a predefined progress threshold value.   
     
     
         12 . The method of  claim 11 , wherein electronically sending the at least one alert comprises:
 generating first, second, and third progress alerts;   electronically sending the first progress alert to a first client computer device associated with a first user account of the learner;   electronically sending the second progress alert to a second client computer device associated with a second user account of the reviewer; and   electronically sending the third progress alert to a third client computer device associated with a third user account of a coach.   
     
     
         13 . The method of  claim 6 , further comprising:
 scanning, by the processor, the plurality of goal ratings to identify a set of goal ratings corresponding to an unprocessed predefined group of learners of a set of predefined groups of learners;   calculating, by the processor, a first mean and a first standard deviation of a first subset of goal ratings of the set of goal ratings, the first subset of goal ratings corresponding to a first predefined category;   calculating, by the processor, a second mean and a second standard deviation of a second subset of goal ratings of the set of goal ratings, the second subset of goal ratings corresponding to a second predefined category;   generating, by the processor, a first chart that is divided into a first plurality of sections, the first plurality of sections corresponding to first goal rating ranges that are bounded based on the first mean and the first standard deviation;   generating, by the processor, a second chart that is divided into a second plurality of sections, the second plurality of sections corresponding to second goal rating ranges that are bounded based on the second mean and the second standard deviation; and   instructing, by the processor, a first client computer device associated with a user account of a learner to display the first chart and the second chart.   
     
     
         14 . The method of  claim 13 , wherein the first chart includes a first indicator corresponding to a first average goal rating for a first goal of the learner over a selected time segment, wherein the second chart includes a second indicator corresponding to a second average goal rating for a second goal of the learner over the selected time segment, wherein the first goal corresponds to the first predefined category, and wherein the second goal corresponds to the second predefined category. 
     
     
         15 . A computer server comprising:
 at least one processor; and   a memory coupled to the at least one processor and configured to store computer-executable instructions that, when executed by the at least one processor, cause the at least one processor to:
 scan a plurality of goal ratings stored in a database at the server device, the plurality of goal ratings corresponding to evaluations of goals of one or more learners; 
 identify that a predefined condition has been met based on the plurality of goal ratings; and 
 electronically send, via at least one electronic communication network, at least one alert to at least one client computer device based on identifying that the predefined condition has been met. 
   
     
     
         16 . The computer server of  claim 15 , wherein the computer-executable instructions further cause the at least one processor to:
 identify that no goal ratings have been submitted by a first rater for a goal of a learner of the one or more learners for a period of time exceeding a predefined inactivity threshold value;   generate first, second, and third inactivity alerts;   electronically send the first inactivity alert to a first client computer device associated with a first user account of the learner;   electronically send the second inactivity alert to a second client computer device associated with a second user account of a reviewer; and   electronically send a third inactivity alert to a third client computer device associated with a third user account of a coach.   
     
     
         17 . The computer server of  claim 15 , wherein the computer-executable instructions further cause the at least one processor to:
 determine that an observation note has been submitted with a goal rating of the plurality of goal ratings, the goal rating corresponding to a goal of the learner of the one or more learners;   generate first and second observation alerts;   electronically send the first observation alert to a first client computer device associated with a first user account of the learner; and   electronically send the second observation alert to a second client computer device associated with a second user account of a coach.   
     
     
         18 . The computer server of  claim 15 , wherein the computer-executable instructions further cause the at least one processor to:
 identify a maximum goal rating of a first set of goal ratings submitted by a second rater within a predetermined time period for a goal of a learner;   identify a minimum goal rating of a second set of goal ratings submitted by the learner for the goal within the predetermined time period;   determine a rating differential by calculating a difference between the minimum goal rating and the maximum goal rating;   determine that the rating differential exceeds a predefined delta threshold value;   generate first and second delta alerts;   electronically send the first delta alert to a first client computer device associated with a first user account of the learner; and   electronically send the second delta alert to a second client computer device associated with a second user account of a coach.   
     
     
         19 . The computer server of  claim 15 , wherein the computer-executable instructions further cause the at least one processor to:
 calculate an average goal rating value by averaging all goal ratings submitted by raters over a predefined progress assessment time period for a goal of a learner;   calculate a progress percentage corresponding to the average goal rating value divided by a target goal rating value set by the learner;   determine that the progress percentage exceeds a predefined progress threshold value;   generate first, second, and third progress alerts;   electronically send the first progress alert to a first client computer device associated with a first user account of the learner;   electronically send the second progress alert to a second client computer device associated with a second user account of the reviewer; and   electronically send the third progress alert to a third client computer device associated with a third user account of a coach.   
     
     
         20 . The computer server of  claim 15 , wherein the computer-executable instructions further cause the at least one processor to:
 scan the plurality of goal ratings to identify to identify a set of goal ratings corresponding to an unprocessed predefined group of learners of a set of predefined groups of learners;   calculate a first mean and a first standard deviation of a first subset of goal ratings of the set of goal ratings, the first subset of goal ratings corresponding to a first predefined category;   calculate a second mean and a second standard deviation of a second subset of goal ratings of the set of goal ratings, the second subset of goal ratings corresponding to a second predefined category;   generate a first chart that is divided into a first plurality of sections, the first plurality of sections corresponding to first goal rating ranges that are bounded based on the first mean and the first standard deviation, the first chart including a first indicator corresponding to a first average goal rating for a first goal of the learner over a selected time segment, the first goal corresponding to the first predefined category;   generate a second chart that is divided into a second plurality of sections, the second plurality of sections corresponding to second goal rating ranges that are bounded based on the second mean and the second standard deviation, the second chart including a second indicator corresponding to a second average goal rating for a second goal of the learner over the selected time segment, the second goal corresponding to the second predefined category; and   instruct a first client computer device associated with a user account of a learner to display the first chart and the second chart.