Systems and methods for mixed reality medical training

Abstract

A system for training a student in medical condition identification and intervention. The system can a wearable mixed reality device and a computing device. The wearable mixed reality device or the computing device has in memory executable instructions for: anchoring a virtual image of at least a portion of a virtual object to a fixed location, the fixed location being in a known relationship to a physical object; displaying the virtual image and one or more virtual critical cues in a 3D augmented reality view on the wearable mixed reality device and in a 2D augmented reality view on the computing device; and assessing a user's identification of the one or more virtual critical cues in the 3D augmented reality view by monitoring the 2D augmented reality view and determining by gaze tracking that one or more of the one or more virtual critical cues was viewed.

What is claimed is: 
     
         1 . A system for training a user, the system comprising:
 a wearable mixed reality device;   a computing device comprising a display screen, the computing device being in communication with the wearable mixed reality device;   a physical object, the physical object being in communication with the computing device; and   one or more fiducial markers positioned on a surface of the physical object;   wherein the wearable mixed reality device or the computing device comprises have in memory executable instructions for:
 anchoring a virtual image of at least a portion of a virtual object to a fixed location, the fixed location being in a known relationship to the one or more fiducial markers, the virtual image being viewable on the wearable mixed reality device and the computing device; 
 storing a scenario comprising parameters for visualization of a condition in the virtual object; 
 selecting the scenario; 
 displaying the virtual image on the mixed reality device and on the display screen of the computing device, the virtual image presenting one or more virtual critical cues; and 
 assessing a user's identification of the one or more virtual critical cues in the 3D augmented reality view by monitoring the display screen of the computing device and determining by gaze tracking that one or more of the one or more virtual critical cues was viewed. 
   
     
     
         2 . The system of  claim 1 , wherein the mixed reality viewing device is head mounted and comprises a holographic processing unit with head-tracking capability. 
     
     
         3 . The system of  claim 1 , wherein the physical object is a mannequin or a portion thereof. 
     
     
         4 . The system of  claim 1 , wherein at least one of the one or more fiducial markers comprise an image of a tattoo. 
     
     
         5 . The system of  claim 1 , wherein the condition is a medical condition and the virtual object is a virtual patient. 
     
     
         6 . The system of  claim 1 , wherein the one or more virtual critical cues comprise cues selected from the group consisting of skin color, vein appearance, bruise appearance, skin swelling, eye movement, mouth movement, blood, saliva, heart rate, pulse oximeter level, and combinations thereof. 
     
     
         7 . The system of  claim 1 , further comprising executable instructions for, upon moving the physical object from a first location to a second location, anchoring the virtual image to the physical object in the second location. 
     
     
         8 . The system of  claim 1 , further comprising executable instructions for commanding the physical object to present movement corresponding to one or more physical critical cues of the time-based state of the condition and assessing the user's identification of the one or more physical critical cues. 
     
     
         9 . The system of  claim 8 , wherein the one or more physical critical cues comprise cues selected from the group consisting of chest movement, pulse rate, and combinations thereof. 
     
     
         10 . The system of  claim 1 , wherein the physical object comprises moveable internal components being responsive to communications from the computing device to present visually perceived external movement of the physical object. 
     
     
         11 . The system of  claim 1 , wherein the scenario comprises a plurality of stages, each stage of the plurality of stages representing a time-based state of the condition. 
     
     
         12 . The system of  claim 11 , wherein the one or more virtual critical cues correspond to the time-based state of the condition. 
     
     
         13 . The system of  claim 1 , wherein, upon selecting the scenario, the selected scenario is transmitted as parameters of the virtual image to the wearable mixed reality viewing device and the physical object. 
     
     
         14 . A system for training a user, the system comprising:
 a wearable mixed reality device;   a computing device; and   a physical object;   wherein the wearable mixed reality device, computing device, and physical object are in communication, and   wherein the wearable mixed reality device or the computing device has in memory executable instructions for:
 anchoring a virtual image of at least a portion of a virtual object to a fixed location, the fixed location being in a known relationship to the physical object; 
 displaying the virtual image and one or more virtual critical cues in a three dimension (3D) augmented reality view on the wearable mixed reality device and in a two dimension (2D) augmented reality view on the computing device; and 
 assessing a user's identification of the one or more virtual critical cues in the 3D augmented reality view by monitoring the 2D augmented reality view and determining by gaze tracking that one or more of the one or more virtual critical cues was viewed. 
   
     
     
         15 . The system of  claim 14 , further comprising executable instructions for:
 storing a scenario in memories of the wearable mixed reality device and computing device, the scenario comprising parameters for visualization of a condition in the virtual object and comprising a plurality of stages, each stage of the plurality of stages representing a time-based state of the condition; and   selecting the scenario, the selected scenario being transmitted as parameters of the virtual image to the wearable mixed reality viewing device and to the physical object.   
     
     
         16 . The system of  claim 14 , further comprising executable instructions for commanding the physical object to present movement corresponding to one or more physical critical cues of a time-based state of a condition and assessing the user's identification of the one or more physical critical cues. 
     
     
         17 . The system of  claim 14 , further comprising executable instructions for, upon moving the physical object from a first location to a second location, anchoring the virtual image to the physical object in the second location. 
     
     
         18 . A system for training a user, the system comprising:
 a wearable mixed reality device; and   a computing device;   wherein the wearable mixed reality device or the computing device has in memory executable instructions for:
 anchoring a virtual image of at least a portion of a virtual object to a fixed location, the fixed location being in a known relationship to a physical object; 
 displaying the virtual image and one or more virtual critical cues in a three dimension (3D) augmented reality view on the wearable mixed reality device and in a two dimension (2D) augmented reality view on the computing device; and 
 assessing a user's identification of the one or more virtual critical cues in the 3D augmented reality view by monitoring the 2D augmented reality view and determining by gaze tracking that one or more of the one or more virtual critical cues was viewed. 
   
     
     
         19 . The system of  claim 18 , further comprising executable instructions for:
 storing a scenario comprising parameters for visualization of a condition in the virtual object and comprising a plurality of stages, each stage of the plurality of stages representing a time-based state of the condition; and   selecting the scenario, the selected scenario being transmitted as parameters of the virtual image to the wearable mixed reality viewing device and to the physical object.   
     
     
         20 . The system of  claim 19 , wherein the scenario comprises a plurality of stages, each stage of the plurality of stages representing a time-based state of the condition, and wherein the one or more virtual critical cues correspond to the time-based state of the condition.