Multi-model member outreach system

Abstract

Member outreach services to be offered to members of a health plan based on predictions generated by a prediction engine using data associated with the members. The prediction engine can include an ensemble of different base predictive models, and the prediction engine can use a weighted combination of base predictions produced by the different base predictive models to generate a final prediction associated with a member. A representative can attempt to contact the member based on an outreach ticket associated with the final prediction. The representative can also provide ticket feedback associated with the outreach, and the prediction engine can use the ticket feedback to adjust the weights associated with the different base predictive models.

What is claimed is: 
     
         1 . A computer-implemented method, comprising:
 training, by one or more processors, different machine learning models to predict one or more metrics, in historical data associated with a set of members, indicating whether members of the set of members would benefit from outreach services;   generating, by the one or more processors and using the different machine learning models, a plurality of base predictions associated with a member, wherein the plurality of base predictions indicates outreach priority levels associated with the member and the outreach services;   generating, by the one or more processors, a final prediction associated with the member by combining the plurality of base predictions based at least in part on weights corresponding to the different machine learning models;   generating, by the one or more processors, an outreach ticket associated with the member based on the final prediction, wherein the outreach ticket causes a representative to at least attempt to offer the outreach services to the member;   receiving, by the one or more processors, ticket feedback associated with the outreach ticket, wherein the ticket feedback indicates at least one score, subjectively assigned by the representative, including one or more of an outreach intervenability score of the member or an interpretability score of the outreach ticket; and   adjusting, by the one or more processors, the weights corresponding to the different machine learning models, based on correlations between the at least one score indicated in the ticket feedback and the outreach priority levels indicated by different base predictions, of the plurality of base predictions, generated by the different machine learning models.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the member is associated with a health plan, and the different machine learning models generate the plurality of base predictions based on at least one of:
 enrollment data associated with the member and the health plan,   claim data associated with the member and the health plan,   demographic data for a geographic location associated with the member,   provider data associated with the geographic location or the member, or   engagement data associated with a use by the member of a website or mobile application associated with the health plan.   
     
     
         3 . The computer-implemented method of  claim 1 , wherein the different machine learning models includes at least two types of machine learning models. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the different machine learning models are trained on different subsets of the historical data. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein the one or more processors use a prediction combination model to adjust the weights based on the correlations. 
     
     
         6 . The computer-implemented method of  claim 1 , wherein:
 the different machine learning models are trained, based on the historical data, until corresponding area under the curve statistics (C-statistics) meet a threshold value, and   the C-statistics indicate probabilities of the different machine learning models correctly predicting which of two members, of the set of members, has healthcare costs that exceed a threshold amount.   
     
     
         7 . The computer-implemented method of  claim 1 , wherein the final prediction indicates at least one of:
 a total cost estimate for the member over a period of time,   an estimated change in costs for the member over the period of time,   a utilization value indicating an estimated number of visits to one or more healthcare providers over the period of time, or   a predicted change to one or more risk metrics associated with the member.   
     
     
         8 . The computer-implemented method of  claim 7 , further comprising:
 determining, by the one or more processors, that the total cost estimate, the estimated change in costs, the utilization value, or the predicted change to the one or more risk metrics exceeds a threshold value; and   generating, by the one or more processors, the outreach ticket based on determining that the total cost estimate, the estimated change in costs, the utilization value, or the predicted change to the one or more risk metrics exceeds the threshold value.   
     
     
         9 . The computer-implemented method of  claim 1 , further comprising providing the outreach ticket, to the representative, in a queue of outreach tickets ordered based on outreach priorities determined based on the final prediction and corresponding final predictions of other outreach tickets in the queue. 
     
     
         10 . The computer-implemented method of  claim 1 , further comprising:
 identifying, by the one or more processors, key variables that most influenced the set of base predictions or the final prediction;   identifying, by the one or more processors, attributes of the member that correspond to the key variables; and   presenting, by the one or more processors, the attributes of the member that correspond to the key variables in the outreach ticket.   
     
     
         11 . The computer-implemented method of  claim 1 , wherein the at least one score indicated in the ticket feedback is based on a Likert scale assessment by the representative. 
     
     
         12 . A system, comprising:
 one or more processors; and   memory storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
 training different machine learning models to predict one or more metrics, in historical data associated with a set of members, indicating whether members of the set of members would benefit from outreach services; 
 generating, using the different machine learning models, a plurality of base predictions associated with a member, wherein the plurality of base predictions indicates outreach priority levels associated with the member and the outreach services; 
 generating a final prediction associated with the member by combining the plurality of base predictions based at least in part on weights corresponding to the different machine learning models; 
 rating an outreach ticket associated with the member based on the final prediction, wherein the outreach ticket causes a representative to at least attempt to offer the outreach services to the member; 
 receiving ticket feedback associated with the outreach ticket, wherein the ticket feedback indicates at least one score, subjectively assigned by the representative, including one or more of an outreach intervenability score of the member or an interpretability score of the outreach ticket; and 
 adjusting the weights corresponding to the different machine learning models, based on correlations between the at least one score indicated in the ticket feedback and the outreach priority levels indicated by different base predictions, of the plurality of base predictions, generated by the different machine learning models. 
   
     
     
         13 . The system of  claim 12 , wherein the member is associated with a health plan, and the different machine learning models generate the plurality of base predictions based on at least one of:
 enrollment data associated with the member and the health plan,   claim data associated with the member and the health plan,   demographic data for a geographic location associated with the member,   provider data associated with the geographic location or the member, or   engagement data associated with a use by the member of a website or mobile application associated with the health plan.   
     
     
         14 . The system of  claim 12 , wherein:
 the different machine learning models are trained, based on the historical data, until corresponding area under the curve statistics (C-statistics) meet a threshold value, and   the C-statistics indicate probabilities of the different machine learning models correctly predicting which of two members, of the set of members, has healthcare costs that exceed a threshold amount.   
     
     
         15 . The system of  claim 12 , wherein the final prediction indicates at least one of:
 a total cost estimate for the member over a period of time,   an estimated change in costs for the member over the period of time,   a utilization value indicating an estimated number of visits to one or more healthcare providers over the period of time, or   a predicted change to one or more risk metrics associated with the member.   
     
     
         16 . The system of  claim 12 , wherein the operations further comprise:
 identifying key variables that most influenced the set of base predictions or the final prediction;   identifying attributes of the member that correspond to the key variables; and   presenting the attributes of the member that correspond to the key variables in the outreach ticket.   
     
     
         17 . One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors of at least one computing device, cause the one or more processors to perform operations comprising:
 training different machine learning models to predict one or more metrics, in historical data associated with a set of members, indicating whether members of the set of members would benefit from outreach services;   generating, using the different machine learning models, a plurality of base predictions associated with a member, wherein the plurality of base predictions indicates outreach priority levels associated with the member and the outreach services;   generating a final prediction associated with the member by combining the plurality of base predictions based at least in part on weights corresponding to the different machine learning models; and   generating a notification associated with the member based on the final prediction,   wherein the weights are determined by a prediction combination model based at least in part on correlations between:
 outreach intervenability scores indicated in subjective representative feedback associated with past member outreach services offered by representatives to other members, and 
 previous outreach priority levels associated with previous base predictions, associated with the other members, generated by the different machine learning models. 
   
     
     
         18 . The one or more non-transitory computer-readable media of  claim 17 , wherein the notification is an outreach ticket that causes a representative to at least attempt to offer the outreach services to the member, and the operations further comprise:
 receiving new ticket feedback, associated with the outreach ticket, indicating a new outreach intervenability score subjectively assigned by the representative in association with the member; and   adjusting the weights, by the prediction combination model, based on correlations between the new outreach intervenability score and the outreach priority levels indicated by different base predictions, of the plurality of base predictions, generated by the different machine learning models.   
     
     
         19 . The one or more non-transitory computer-readable media of  claim 17 , wherein the notification is an automated notification, and the operations further comprise sending the automated notification to a computing device associated with the member. 
     
     
         20 . The one or more non-transitory computer-readable media of  claim 17 , wherein the final prediction indicates at least one of:
 a total cost estimate for the member over a period of time,   an estimated change in costs for the member over the period of time,   a utilization value indicating an estimated number of visits to one or more healthcare providers over the period of time, or   a predicted change to one or more risk metrics associated with the member.