Medical treatment planning via sequential games

Abstract

A method and system for identifying a treatment plan identifies a description of a sequential game. The game is associated with treatment of a medical condition by a course of treatment or drug design. The description may include one or more possible treatment actions that a treater can take to treat the medical condition, and one or more possible medical condition actions that the medical condition can take. The system may develop a model for the sequential game, wherein the model represents implementation of the possible treatment actions and the possible medical condition actions in one or more sequences. The system may solve the model to generate a treatment plan for the medical condition, wherein the treatment plan includes a set of possible treatment actions.

What is claimed is: 
     
         1 . A system for developing a course of treatment for a medical condition, comprising:
 one or more processors; and   a non-transitory, computer-readable memory comprising one or more programming instructions that, when executed, cause one or more of the processors to:
 identify a description of a sequential game, wherein the game is associated with treatment of a medical condition, wherein the description comprises:
 one or more possible treatment actions that a treater can take to treat the medical condition, and 
 one or more possible medical condition actions that the medical condition can take; 
 
 develop a model for the sequential game, wherein the model represents implementation of the possible treatment actions and the possible medical condition actions in one or more sequences; and 
 solve the model to generate a treatment plan for the medical condition, wherein the treatment plan comprises a set of possible treatment actions. 
   
     
     
         2 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve a model comprise one or more programming instructions that, when executed, cause the one or more processors to generate one or more contingent plans in the model. 
     
     
         3 . The system of  claim 1 , wherein the plan comprises randomization via behavioral or mixed strategies. 
     
     
         4 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve the model to generate a treatment plan for the medical condition comprise one or more programming instructions that, when executed, cause the one or more processors to:
 generate a treatment plan comprising one or more traps, where the medical condition is likely to take actions so as to fall into a trap that causes the medical condition to go into one or more of the following:
 a state in which the medical condition may be more easily treated; 
 a state in which the medical condition is less virulent; or 
 a state in which the medical condition is less contagious; or 
 a state from where the medical condition is less likely to evolve into a harmful state. 
   
     
     
         5 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve the model comprise one or more programming instructions that, when executed, cause the one or more processors to:
 apply an opponent model in which the medical condition is able to look ahead at most a set number of steps in the game; and   create a path for the medical condition in which a sequence of steps includes one or more steps within the set number that are attractive to the medical condition, and at least one step beyond the set number that is associated with a state of the medical condition that is better for the treater, a patient with the medical condition, or both.   
     
     
         6 . The system of  claim 1 , wherein the one or more possible treatment actions that a treater can take to treat the medical condition comprise one or more possible treatment actions that a treater can take to treat the medical condition at an individual level, a molecular level, or a population level. 
     
     
         7 . The system of  claim 1 , wherein:
 the one or more possible treatment actions comprise one or more possible treatment actions to treat the medical condition at a molecular level via a de novo drug.   
     
     
         8 . The system of  claim 1 , wherein the one or more possible treatment actions that to treater can take to treat the medical condition comprise one or more of the following:
 prescribing or administering one or more drugs to a patient having the medical condition;   performing a surgical procedure on the patient having the medical condition;   applying a therapy to the patient;   prescribing a lifestyle change to the patient;   admitting the patient to a treatment facility;   releasing the patient from the treatment facility;   taking one or more measurements of the patient; or   taking no action.   
     
     
         9 . The system of  claim 1 , wherein
 the description of the sequential game further comprises one or more possible nature actions that a nature player may take relating to treatment of the medical condition, wherein each possible nature action is associated with a probability; and   the model represents occurrence of the possible treatment actions, the possible medical condition actions and the possible nature actions in the one or more sequences.   
     
     
         10 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve the model comprise one or more programming instructions that, when executed, cause the one or more processors to solve the model using one or more game theory solution concepts and one or more utilities that are associated with outcomes, intermediate states, or transitions in sequential game play. 
     
     
         11 . The system of  claim 10 , wherein the one or more utilities comprise as function of a measurement of one or more of the following:
 health of a patient with the medical condition;   as cost to the patient, the treater, or a third party payor; or   a current state of the medical condition.   
     
     
         12 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve the model comprise one or more programming instructions that when executed, cause the one or more processors to:
 implement an opponent modeling technique; or   implement an opponent exploitation technique.   
     
     
         13 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve the model comprise one or more programming instructions that, when executed, cause the one or more processors to:
 exploit an opponent as an opponent model is improved over time based on experience.   
     
     
         14 . The system of  claim 1 , wherein the computer-readable memory further comprises one or more programming instructions that, when executed, cause the one or more processors to present at least a portion of the treatment plan to a user via a user interface. 
     
     
         15 . The system of  claim 1 , wherein the computer-readable memory further comprises one or more programming instructions that, when executed, cause the one or more processors to use information learned while using results of the model to develop an updated model. 
     
     
         16 . The system of  claim 1 , wherein the one or more programming instructions that, when executed, cause the one or more processors to solve the model comprise one or more programming instructions that, when executed, cause the one or more processors to compute a best-response strategy to an opponent model using stochastic programming. 
     
     
         17 . The system of  claim 16 , wherein the use of stochastic programming comprises use of one or more of the following:
 sample trajectory-based optimization, or   a policy gradient algorithm.   
     
     
         18 . A method of developing a course of treatment for a medical condition, comprising, by one or more processors:
 identifying a description of a sequential game, wherein the game is associated with treatment of a medical condition, wherein the description comprises:
 one or more possible treatment actions that a treater can take to treat the medical condition, and 
 one or more possible medical condition actions that the medical condition can take; 
   developing a model for the sequential game, wherein the model represents implementation of the possible treatment actions and the possible medical condition actions in one or more sequences; and   solving the model to generate a treatment plan for the medical condition, wherein the treatment plan comprises a set of possible treatment actions.   
     
     
         19 . The method of  claim 18 , wherein solving the model comprises:
 implementing an opponent modeling technique; or   implementing an opponent exploitation technique.   
     
     
         20 . The method of  claim 1 , wherein solving the model comprises using one or more game theory solution concepts and one or more utilities that are associated with outcomes, intermediate states, or transitions in sequential game play, wherein the one or more utilities comprise a function of a measurement of one or more of the following:
 health of a patient with the medical condition;   a cost to the patient, the treater, or a third party payor; or   a current state of the medical condition.