Extended reality for moving platforms

Abstract

Implementations of the subject technology provide extended reality display devices that can be used on and/or off of a moving platform. Systems and methods are disclosed for separating out the motion of the moving platform from other motions of the device so that virtual content can be displayed without erroneous motions caused by the motion of the moving platform. The subject technology can provide extended reality settings on any suitable moving platform such as in a car, a watercraft, an aircraft, a train, or any other vehicle.

What is claimed is: 
     
         1 . A device, comprising:
 a plurality of inertial measurement unit sensors; and   one or more processors configured to:
 determine, using the plurality of inertial measurement unit sensors, that device motion of the device includes a first component associated with a motion of a moving platform and a second component that is separate from the motion of the moving platform; and 
 display virtual content at a fixed location relative to the moving platform, using less than all of the plurality of inertial measurement unit sensors to track the second component of the device motion that is separate from the motion of the moving platform. 
   
     
     
         2 . The device of  claim 1 , further comprising:
 an optical sensor; and   a depth sensor,   wherein the one or more processors are configured to display the virtual content at the fixed location relative to the moving platform using the optical sensor and the depth sensor, without further input from the plurality of inertial measurement unit sensors.   
     
     
         3 . The device of  claim 1 , wherein the second component of the device motion that is separate from the motion of the moving platform comprises a rotational motion of the device. 
     
     
         4 . The device of  claim 3 , wherein the second component of the device motion that is separate from the motion of the moving platform includes a translational motion of the device relative to the moving platform. 
     
     
         5 . The device of  claim 1 , wherein the one or more processors are configured to display the virtual content at the fixed location relative to the moving platform by:
 detecting an object with one or more sensors of the device;   determining that the object has an object motion that is equal to the motion of the moving platform; and   anchoring the virtual content to the object.   
     
     
         6 . The device of  claim 5 , wherein the one or more processors are further configured to:
 detect a motion of the object relative to the moving platform and relative to the device; and   continue anchoring the virtual content to the object based on the detected motion of the object.   
     
     
         7 . The device of  claim 1 , wherein the one or more processors are further configured to:
 detect a portion of a physical setting having a motion that is different from the motion of the moving platform and different from the device motion of the device.   
     
     
         8 . The device of  claim 7 , wherein the one or more processors are further configured to:
 determine that the motion of the portion of the physical setting is different from the motion of the moving platform and different from the device motion of the device by at least a threshold; and   identify the portion of the physical setting as ineligible for anchoring of virtual content.   
     
     
         9 . The device of  claim 8 , wherein the one or more processors are further configured to display additional virtual content overlaid on the portion of the physical setting and fixed relative to the moving platform to block a view of the portion of the physical setting. 
     
     
         10 . The device of  claim 7 , wherein the one or more processors are configured to detect the portion of the physical setting having the motion that is different from the motion of the moving platform and different from the device motion of the device by detecting an optical flow associated with the portion of the physical setting using an optical sensor of the device. 
     
     
         11 . The device of  claim 7 , wherein the one or more processors are further configured to:
 receive an input from a user indicating a repositioning of the virtual content; and   responsive to the input, display the virtual content anchored to the portion of the physical setting.   
     
     
         12 . The device of  claim 11 , wherein the portion of the physical setting includes an object that is fixed to a fixed reference frame external to the moving platform and the device. 
     
     
         13 . The device of  claim 1 , wherein the one or more processors are further configured to track the motion of the moving platform using a first simultaneous localization and mapping system and to track the second component that is separate from the motion of the moving platform using a second simultaneous localization and mapping system. 
     
     
         14 . The device of  claim 13 , wherein the second simultaneous localization and mapping system comprises one or more sensors of the device. 
     
     
         15 . The device of  claim 14 , wherein the first simultaneous localization and mapping system comprises at least one additional sensor on the moving platform and configured to provide platform motion information to the one or more processors of the device. 
     
     
         16 . A method, comprising:
 determining, using a plurality of inertial measurement unit sensors of a device, that device motion of the device includes a first component associated with a motion of a moving platform and a second component that is separate from the motion of the moving platform; and   displaying, with a display of the device, virtual content at a fixed location relative to the moving platform, using less than all of the plurality of inertial measurement unit sensors to track the second component of the device motion that is separate from the motion of the moving platform.   
     
     
         17 . The method of  claim 16 , wherein displaying the virtual content at the fixed location relative to the moving platform comprises:
 detecting an object that is stationary relative to the moving platform using the one or more sensors;   anchoring the virtual content to the object;   detecting a motion of the object relative to the moving platform and relative to device; and   continuing to anchor the virtual content to the object based on the detected motion of the object.   
     
     
         18 . The method of  claim 16 , wherein the plurality of inertial measurement unit sensors comprises at least an accelerometer, a gyroscope, and a magnetometer, and wherein less than all of the plurality of inertial measurement unit sensors comprises a subset of the accelerometer, the gyroscope, and the magnetometer. 
     
     
         19 . A non-transitory computer-readable medium comprising instructions, which when executed by a computing device, cause the computing device to:
 determine, using a plurality of inertial measurement unit sensors of the computing device, that device motion of the computing device includes a first component associated with a motion of a moving platform and a second component that is separate from the motion of the moving platform; and   display virtual content at a fixed location relative to the moving platform, using less than all of the plurality of inertial measurement unit sensors to track the second component of the device motion that is separate from the motion of the moving platform.   
     
     
         20 . The non-transitory computer-readable medium of  claim 19 , wherein the virtual content is displayed as part of a virtual setting that entirely blocks a physical setting from a view of a user, or to appear as an anchoring location in the physical setting.