Augmented and virtual reality simulator for professional and educational training

Abstract

A method and apparatus for an augmented reality simulator for professional and educational training is provided. The simulator provides a training environment spanning one or more physical locations in which one or more virtual avatars representing purely virtual objects or persons or real physical objects or persons which are located at a different physical location are projected into the physical space. The avatars are interactive with other avatars and real objects or persons and update over time or in response to actions taken by other real or virtual elements, or based on predefined instructions. Sensors and devices are used to detect the locations of and actions taken by real persons or real objects and this sensed data is used to evolve the state of the simulation and avatars based on predefined instructions and programs and update the view of all participants.

What is claimed is: 
     
         1 . A method comprising;
 detecting, using at least one sensor, an occurrence of an action within an augmented or virtual reality environment;   recording, using at least one computer device, the occurrence of an action within an augmented or virtual reality environment;   comparing a record of the action to a performance metric that identifies a pre-determined set of goal actions;   generating, based at least in part on said comparing, a performance evaluation that includes one or more measures that indicate a performance of at least one participant in the augmented or virtual reality environment.   
     
     
         2 . The method according to  claim 1 , wherein the action by the at least one participant that is detected and recorded includes at least one of a body movement, direct input of data into the at least one computer device, at least one vocalization, or an expression of specific ideas via body movement or vocalization. 
     
     
         3 . The method of  claim 1 , wherein the action comprises at least one of performing a medical procedure on a virtual avatar projected into the augmented or virtual reality environment or a vocalization diagnosing a medical condition of the virtual avatar projected into the augmented or virtual reality environment. 
     
     
         4 . The method according to  claim 1 , wherein the actions recorded within the augmented or virtual reality environment are appended to a record of the augmented or virtual reality environment session. 
     
     
         5 . The method according to  claim 4 , wherein the record of the augmented or virtual reality environment session is comprised of a record of an execution of a file containing an instruction set controlling the augmented or virtual reality environment, timing data regarding when instructions were executed and events occurred, a plurality of events occurring during execution of the instruction set, a plurality of data collected from a plurality of participant apparatuses, and video representing the augmented or virtual reality environment from a perspective of at least one participant or a virtual third party perspective. 
     
     
         6 . The method according to  claim 5 , wherein an event in the plurality of events corresponds to at least one change to the augmented or virtual reality environment. 
     
     
         7 . The method according to  claim 1 , wherein the performance evaluation includes a grade that is determined based at least in part on a length of time to progress between different states in a simulation of a medical operation. 
     
     
         8 . The method according to  claim 1 , wherein the predetermined set of goal actions are defined by an author of an instruction set for executing a simulation of a medical operation within the virtual or augmented reality environment. 
     
     
         9 . The method according to  claim 1 , wherein detecting, by the sensor, occurrence of the action comprises detecting placement of a real object relative to a virtual avatar projected in the virtual or augmented reality environment. 
     
     
         10 . One or more non-transitory computer-readable media storing instructions which, when executed by one or more hardware processors, cause operations comprising:
 detecting, using at least one sensor, an occurrence of an action within an augmented or virtual reality environment;   recording, using at least one computer device, the occurrence of an action within an augmented or virtual reality environment;   comparing a record of the action to a performance metric that identifies a pre-determined set of goal actions;   generating, based at least in part on said comparing, a performance evaluation that includes one or more measures that indicate a performance of at least one participant in the augmented or virtual reality environment.   
     
     
         11 . The media according to  claim 10 , wherein the action by the at least one participant that is detected and recorded includes at least one of a body movement, direct input of data into the at least one computer device, at least one vocalization, or an expression of specific ideas via body movement or vocalization. 
     
     
         12 . The media of  claim 10 , wherein the action comprises at least one of performing a medical procedure on a virtual avatar projected into the augmented or virtual reality environment or a vocalization diagnosing a medical condition of the virtual avatar projected into the augmented or virtual reality environment. 
     
     
         13 . The media according to  claim 10 , wherein the actions recorded within the augmented or virtual reality environment are appended to a record of the augmented or virtual reality environment session. 
     
     
         14 . The media according to  claim 13 , wherein the record of the augmented or virtual reality environment session is comprised of a record of an execution of a file containing an instruction set controlling the augmented or virtual reality environment, timing data regarding when instructions were executed and events occurred, a plurality of events occurring during execution of the instruction set, a plurality of data collected from a plurality of participant apparatuses, and video representing the augmented or virtual reality environment from a perspective of at least one participant or a virtual third party perspective. 
     
     
         15 . The media according to  claim 14 , wherein an event in the plurality of events corresponds to at least one change to the augmented or virtual reality environment. 
     
     
         16 . The media according to  claim 10 , wherein the performance evaluation includes a grade that is determined based at least in part on a length of time to progress between different states in a simulation of a medical operation. 
     
     
         17 . The media according to  claim 10 , wherein the predetermined set of goal actions are defined by an author of an instruction set for executing a simulation of a medical operation within the virtual or augmented reality environment. 
     
     
         18 . The media according to  claim 10 , wherein detecting, by the sensor, occurrence of the action comprises detecting placement of a real object relative to a virtual avatar projected in the virtual or augmented reality environment. 
     
     
         19 . A system comprising:
 one or more hardware processors;   one or more non-transitory computer-readable media storing instructions which, when executed by the one or more hardware processors, cause:
 detecting, using at least one sensor, an occurrence of an action within an augmented or virtual reality environment; 
 recording, using at least one computer device, the occurrence of an action within an augmented or virtual reality environment; 
 comparing a record of the action to a performance metric that identifies a pre-determined set of goal actions; 
 generating, based at least in part on said comparing, a performance evaluation that includes one or more measures that indicate a performance of at least one participant in the augmented or virtual reality environment. 
   
     
     
         20 . The system of  claim 19 , wherein the action by the at least one participant that is detected and recorded includes at least one of a body movement, direct input of data into the at least one computer device, at least one vocalization, or an expression of specific ideas via body movement or vocalization.