Virtual shoot wall with 3d space and avatars reactive to user fire, motion, and gaze direction

Abstract

A simulator system includes functionality for dynamically tracking position and orientation of one or more simulation participants and objects as they move throughout a capture volume using an array of motion capture video cameras so that two- or three-dimensional (“2D” and “3D”) views of a virtual environment, which are unique to each participant's point of view, may be generated by the system and rendered on a display. In 3D and/or multi-participant usage scenarios, the unique views are decoded from a commonly utilized display by equipping the participants with glasses that are configured with shutter lenses, polarizing filters, or a combination of both. The object tracking supports the provision and use of an optical signaling capability that may be added to an object so that manipulation of the object by the participant can be communicated to the simulator system over the optical communications path that is enabled by use of the video cameras.

1 . A method for operating a simulation supported on a simulator, the method comprising the steps of:
 tracking a participant in the simulation to determine at least one of position, orientation, or motion of the participant within a capture volume, the capture volume being monitored by an optical motion capture system that is configured to capture positions of participant markers within the capture volume, the participant markers being positioned at known locations on the participant;   configuring an object with i) object markers at known locations so that the optical motion capture system can capture positions of the object markers within the capture volume and ii) at least one participant-actuated light source that is disposed at a known location on the object and monitored by the optical capture system so as to implement an optical communications path between the object and the optical motion capture system over which a signal may be transmitted via actuation of the light source;   tracking the object to determine at least one of position, orientation, or motion of the object within a capture volume; and   dynamically generating a virtual environment utilized by the simulation, the virtual environment being generated from the participant's point of view responsively to the participant tracking and further being generated responsively to the object tracking and signal transmitted over the optical communications path.   
     
     
         2 . The method of  claim 1  including a further step of rendering the virtual environment onto a display, the display being one of shoot wall or CAVE. 
     
     
         3 . The method of  claim 1  in which the object comprises a simulated weapon and the user-actuated light source is operatively coupled to a trigger on the weapon, the light source being operated in response to a trigger pull. 
     
     
         4 . The method of  claim 3  in which the light source is an IR light source and the signal is indicative of the weapon being fired. 
     
     
         5 . The method of  claim 3  including a further step of determining a trajectory of a simulated discharge of a round from the weapon using the object tracking. 
     
     
         6 . The method of  claim 1  in which the optical motion capture system utilizes an array of video cameras, each of the video cameras including one or more IR light sources. 
     
     
         7 . The method of  claim 1  in which the virtual environment includes one or more avatars that are responsive to the tracked participant, the avatars being configured to dynamically change gaze or weapon aim in response to the position, orientation, or motion of the participant within the capture volume. 
     
     
         8 . The method of  claim 1  in which the participant tracking comprises tracking the participant's head. 
     
     
         9 . A computer-implemented method for providing a shoot wall simulation, the method comprising the steps of:
 tracking a position within a capture volume of each of one or more participants in the shoot wall simulation using a motion capture system that is configured to monitor the capture volume;   generating a unique view for each of one or more participants, each unique view being taken from a point of view of the respective participant as the participant moves within the capture volume;   superimposing the unique views onto a display device that is commonly utilized by each of the one or more participants;   tracking a position of a weapon associated with one or more of the participants; and   detecting operation of the weapon using the motion capture system, the detecting comprising monitoring actuation of a light affixed to the weapon, the light being actuated when the weapon is fired.   
     
     
         10 . The computer-implemented method of  claim 9  in which the unique views are encoded as 3D views with left-eye and right-eye images. 
     
     
         11 . The computer-implemented method of  claim 10  in which the left-eye and right-eye images are decoded using one of LCD shutter glasses or polarizing filter glasses. 
     
     
         12 . The computer-implemented method of  claim 9  in which the commonly utilized display device utilizes multiple walls in a CAVE configuration. 
     
     
         13 . The computer-implemented method of  claim 9  in which the weapon utilizes at least two markers disposed substantially along a long axis defined by the barrel of the weapon. 
     
     
         14 . The computer-implemented method of  claim 9  in which the markers comprise substantially spherical retro-reflectors. 
     
     
         15 . One or more computer-readable storage media containing instructions which, when executed by one or more processors disposed in a computing device, implement a simulator system, the instructions being logically grouped in modules, the modules comprising:
 a camera module for interfacing with an array of optical motion capture video cameras, the array being configured for optically monitoring a capture volume and for receiving captured images of tracked simulation participants and weapons associated with respective participants;   a head tracking module for determining a position of a head of one or more of the tracked simulation participants within the capture volume using the captured images;   a weapon tracking module for determining a position of one or more tracked weapons within the capture volume using the captured images; and   a virtual environment generation module for generating a virtual environment supported by the simulator system, the virtual environment being corrected for parallax distortion and trajectory parallax.   
     
     
         16 . The one or more computer-readable storage media of  claim 15  further comprising a virtual environment rendering module for rendering the generated virtual environment onto a display. 
     
     
         17 . The one or more computer-readable storage media of  claim 15  in which the parallax distortion correction comprises generating a unique view for each participant based on each participant's point of view within the capture volume. 
     
     
         18 . The one or more computer-readable storage media of  claim 15  in which the trajectory parallax correction comprises determining a trajectory of a round fired from one or more of the weapons using the tracked position of the one or more weapons. 
     
     
         19 . The one or more computer-readable storage media of  claim 15  in which the virtual environment generation module generates avatars that are responsive to the position of the participants. 
     
     
         20 . The one or more computer-readable storage media of  claim 16  in which the rendering is performed in 3D.