Head-mounted display system

Abstract

A head-mounted display system is disclosed herein. In one or more embodiments, a system for body pose estimation in a virtual reality or augmented reality environment includes one or more egocentric cameras coupled to a user configured to capture video data from a perspective of the user; an instrumented flooring system including one or more sensors configured to detect foot placement and/or movement of the user, and generate sensor output data based on the detected foot placement and/or movement of the user; and at least one data processing device being configured to receive and integrate the video data from the one or more egocentric cameras and the sensor output data from the instrumented flooring system using sensor fusion algorithms, thereby generating a comprehensive body pose estimation for the user.

The invention claimed is: 
     
         1 . A system for body pose estimation in a virtual reality or augmented reality environment, comprising:
 one or more egocentric cameras coupled to a user, the one or more egocentric cameras configured to capture video data from a perspective of the user;   an instrumented flooring system, the instrumented flooring system including one or more sensors configured to detect foot placement and/or movement of the user, and generate sensor output data based on the detected foot placement and/or movement of the user; and   at least one data processing device, the at least one data processing device operatively coupled to the one or more egocentric cameras and the one or more sensors of the instrumented flooring system, the at least one data processing device being configured to receive and integrate the video data from the one or more egocentric cameras and the sensor output data from the instrumented flooring system using sensor fusion algorithms, thereby generating a comprehensive body pose estimation for the user.   
     
     
         2 . The system according to  claim 1 , wherein the one or more egocentric cameras coupled to the user comprise a plurality of egocentric cameras that are attached to different parts of the body of the user to capture a range of perspectives and movements of the user, the different parts of the body of the user being selected from a group consisting of: (i) a head of the user, (ii) a chest of the user, (iii) one or more arms of the user, (iv) one or more legs of the user, and (v) combinations thereof. 
     
     
         3 . The system according to  claim 1 , wherein the one or more sensors of the instrumented flooring system comprise a plurality of sensors configured to detect a weight distribution, foot placement, and/or gait patterns of the user, and the plurality of sensors are selected from a group consisting of: (i) pressure sensors, (ii) capacitive sensors, (iii) load cells, and (iv) combinations thereof. 
     
     
         4 . The system according to  claim 1 , wherein the at least one data processing device is further configured to execute synchronization protocols for aligning the video data from the one or more egocentric cameras and the sensor output data from the instrumented flooring system in a common coordinate system, thereby ensuring coherent and accurate pose estimation. 
     
     
         5 . The system according to  claim 1 , wherein the at least one data processing device is further configured to perform real-time processing of the video data from the one or more egocentric cameras and the sensor output data from the instrumented flooring system so as to provide quasi-instantaneous updates to the virtual reality or augmented reality environment based on body movements of the user. 
     
     
         6 . The system according to  claim 1 , wherein the sensor fusion algorithms executed by the at least one data processing device include error correction mechanisms to compensate for potential drift in the video data from the one or more egocentric cameras using stable reference points from the instrumented flooring system. 
     
     
         7 . The system according to  claim 1 , wherein the at least one data processing device is further configured to dynamically update the virtual reality or augmented reality environment based on the body pose estimation for the user, thereby enhancing user interaction and immersion in the virtual reality or augmented reality environment. 
     
     
         8 . The system according to  claim 1 , wherein the sensor fusion algorithms executed by the at least one data processing device include feedback loops to continuously refine the body pose estimation for the user by compensating for drift in the video data from the one or more egocentric cameras using reference points from the instrumented flooring system. 
     
     
         9 . The system according to  claim 1 , wherein the at least one data processing device is further configured to provide real-time feedback and interaction capabilities in the virtual reality or augmented reality environment based on body movements of the user. 
     
     
         10 . The system according to  claim 9 , wherein the at least one data processing device is further configured to dynamically adjust training scenarios for the user in response to the body movements of the user, thereby enhancing the effectiveness of a training session.