Virtual Reality System with Modeling Poses of Tracked Objects by Predicting Sensor Data

Abstract

A platform system receives sensor data describing the state and orientation of a tracked object and models the pose of the tracked object to determine user interactions with the platform system. To ensure that incorrect sensor data due to a saturation event or connection loss does not impact user experience, the platform system identifies regions for correction in sensor data streams based on the sensor data being at or above a saturation limit or not being received. The platform system predicts sensor data for an identified region of correction by applying a fit corresponding to points adjacent to the region for correction and determining predicted sensor data using the applied fit. The predicted sensor data is used to correct the modeled pose for the tracked object.

What is claimed is: 
     
         1 . A virtual reality system comprising:
 a sensor data store circuit configured to store sensor data associated with movements and positions of a tracked controller held by a user and a headset worn by the user;   a pose modeling circuit configured to model a pose of the tracked controller based on the sensor data;   a sensor data correction circuit configured to:
 identify a region for correction in the sensor data, 
 determine a piecewise linear fit based on identifying that the sensor data are collected by at least one accelerometer, and 
 apply the determined piecewise linear fit corresponding to points adjacent to the region for correction using a portion of the sensor data to determine predicted sensor data for the region for correction; 
   a pose correction circuit configured to perform a gradual modification of the modeled pose using the predicted sensor data to generate a modified modeled pose; and   a processor configured to:
 identify actions and interactions made by the user with the virtual reality system, and 
 render a state of the tracked controller in accordance with a real-world position of the tracked controller for presentation to the user by the virtual reality system. 
   
     
     
         2 . The virtual reality system of  claim 1 , wherein the pose modeling circuit is further configured to:
 identify an initial pose of the tracked controller based on a first portion of the sensor data; and   update the initial pose to model the pose of the tracked controller based on a second portion of the sensor data.   
     
     
         3 . The virtual reality system of  claim 1 , wherein the sensor data correction circuit is further configured to:
 identify a saturation limit for a plurality of sensors including the at least one accelerometer; and   identify the region in the sensor data where a subset of the sensor data is at or above the saturation limit.   
     
     
         4 . The virtual reality system of  claim 1 , wherein the sensor data correction circuit is further configured to:
 identify the region in the sensor data wherein no sensor data is received.   
     
     
         5 . The virtual reality system of  claim 1 , wherein the sensor data correction circuit is further configured to:
 determine a slope of readings adjacent to the region for correction; and   apply the determined piecewise linear fit corresponding to the determined slope.   
     
     
         6 . The virtual reality system of  claim 5 , wherein the sensor data correction circuit is further configured to determine the slope based on two of the readings adjacent to the region for correction. 
     
     
         7 . The virtual reality system of  claim 1 , wherein the sensor data correction circuit is further configured to:
 determine a number and a frequency of predicted sensor points in the predicted sensor data based on a frequency of the points adjacent to the region for correction and a duration of time of the region for correction.   
     
     
         8 . The virtual reality system of  claim 1 , wherein the pose correction circuit is further configured to perform the gradual modification of the modeled pose by applying a series of modifications to the modeled pose over time. 
     
     
         9 . The virtual reality system of  claim 1 , wherein the sensor data are collected by a first set of sensors associated with the tracked controller and a second set of sensors associated with the headset. 
     
     
         10 . The virtual reality system of  claim 9 , wherein the first set of sensors comprises the at least one accelerometer. 
     
     
         11 . The virtual reality system of  claim 1 , further comprising a sensor calibration circuit configured to apply calibration parameters to the sensor data prior to storing the sensor data in the sensor data store circuit. 
     
     
         12 . The virtual reality system of  claim 1 , wherein the user interacts with the virtual reality system using the tracked controller and the headset. 
     
     
         13 . The virtual reality system of  claim 1 , wherein the tracked controller and the headset communicate with the virtual reality system via a wireless communication. 
     
     
         14 . A virtual reality system comprising:
 a sensor data store circuit configured to store sensor data associated with movements and positions of a controller held by a user of the virtual reality system;   a pose modeling circuit configured to model a pose of the controller based on the sensor data;   a sensor data correction circuit configured to:
 identify a region for correction in the sensor data, 
 determine a piecewise linear fit based on identifying that the sensor data are collected by at least one accelerometer, and 
 apply the determined piecewise linear fit corresponding to points adjacent to the region for correction using a portion of the sensor data to determine predicted sensor data for the region for correction; 
   a pose correction circuit configured to perform a gradual modification of the modeled pose using the predicted sensor data to generate a modified modeled pose; and   a processor configured to:
 identify actions and interactions made by the user with the virtual reality system, and 
 render a state of the controller in accordance with a real-world position of the controller for presentation to the user by the virtual reality system. 
   
     
     
         15 . The virtual reality system of  claim 14 , wherein the sensor data correction circuit is further configured to:
 determine a slope of readings adjacent to the region for correction; and   apply the determined piecewise linear fit corresponding to the determined slope.   
     
     
         16 . The virtual reality system of  claim 14 , wherein the sensor data correction circuit is further configured to:
 determine a number and a frequency of predicted sensor points in the predicted sensor data based on a frequency of the points adjacent to the region for correction and a duration of time of the region for correction.   
     
     
         17 . The virtual reality system of  claim 1 , wherein the sensor data are collected by a plurality of sensors associated with the controller. 
     
     
         18 . The virtual reality system of  claim 17 , wherein the plurality of sensors comprises the at least one accelerometer. 
     
     
         19 . The virtual reality system of  claim 14 , wherein the user interacts with the virtual reality system using the controller, and the controller communicates with the virtual reality system via a wireless connection. 
     
     
         20 . A virtual reality system comprising:
 a headset worn by a user;   a sensor data store circuit configured to store sensor data associated with movements and positions of the headset and a tracked controller held by the user;   a pose modeling circuit configured to model a pose of the tracked controller based on the sensor data;   a sensor data correction circuit configured to:
 identify a region for correction in the sensor data, 
 determine a piecewise linear fit based on identifying that the sensor data are collected by at least one accelerometer, and 
 apply the determined piecewise linear fit corresponding to points adjacent to the region for correction using a portion of the sensor data to determine predicted sensor data for the region for correction; 
   a pose correction circuit configured to perform a gradual modification of the modeled pose using the predicted sensor data to generate a modified modeled pose; and   a processor configured to:
 identify actions and interactions made by the user with the virtual reality system, and 
 render a state of the tracked controller in accordance with a real-world position of the tracked controller for presentation to the user by the virtual reality system.