System for Occlusion Adjustment for In-Vehicle Augmented Reality Systems

Abstract

The disclosure includes embodiments for providing occlusion adjustment for an in-vehicle augmented reality system. A system may include a three-dimensional heads-up display unit (“3D HUD”) installed in a vehicle. The system may include a memory storing instructions that, when executed, cause the system to: display, on the 3D HUD, a first graphic that is viewable by a driver of the vehicle as overlaying the interested object when the driver is looking at the 3D HUD; determine that at least a portion of the interested object is occluded by an occluding object; turn off the first graphic so that the first graphic is not displayed on the 3D HUD; and display, on the 3D HUD, a second graphic that does not overlay the occluding object and visually indicates to the driver the location of the interested object behind the occluding object when the driver is looking at the 3D HUD.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 receiving sensor data from external sensors associated with a roadside unit, the sensor data describing a first position of an interested object in an environment external to a vehicle;   determining an importance value associated with the interested object;   displaying, on an electronic display, an initial graphic based on the first position of the interested object;   determining that at least a portion of the interested object is occluded by an occluding object;   determining that the importance value meets or exceeds an importance threshold;   responsive to a determination that the importance value meets or exceeds the importance threshold, turning off the initial graphic so that the initial graphic is not displayed on the electronic display; and   displaying, on the electronic display, a non-occluding graphic that does not overlay the occluding object or the interested object and that visually indicates to a driver of the vehicle that the interested object is located behind the occluding object when the driver is looking at the electronic display.   
     
     
         2 . The method of  claim 1 , wherein determining, based on the importance value, that the interested object is not allowed to be occluded by the occluding object includes determining that the importance value fails to meet or exceed the importance threshold. 
     
     
         3 . The method of  claim 1 , wherein the non-occluding graphic is an arrow located above the interested object that indicates to the driver a location of the interested object behind the occluding object. 
     
     
         4 . The method of  claim 1 , wherein the sensor data describes a type of object and determining the importance value is based on the type of object. 
     
     
         5 . The method of  claim 1 , further comprising tracking motion of the interested object based on the sensor data from the external sensors and based on sensor data from one or more vehicle sensors. 
     
     
         6 . The method of  claim 5 , wherein the one or more vehicle sensors includes a camera that tracks the motion of the interested object. 
     
     
         7 . The method of  claim 5 , wherein the vehicle sensors detect the interested object moving from the first position to a second position and the method further comprises repositioning a location of the non-occluding graphic as displayed on the electronic display to correspond to the second position wherein the non-occluding graphic was previously displayed on the electronic display to correspond to the first position. 
     
     
         8 . A computer program product comprising a non-transitory computer-usable medium including a computer-readable program, wherein the computer-readable program when executed on a computer of a vehicle causes the computer to:
 receive sensor data from external sensors associated with a roadside unit, the sensor data describing a first position of an interested object in an environment external to the vehicle;   determine an importance value associated with the interested object;   display an initial graphic on an electronic display based on the first position of the interested object;   determine that at least a portion of the interested object is occluded by an occluding object;   determine that the importance value meets or exceeds an importance threshold;   responsive to a determination that the importance value meets or exceeds the importance threshold, turn off the initial graphic so that the initial graphic is not displayed on the electronic display; and   display, on the electronic display, a non-occluding graphic that does not overlay the occluding object or the interested object and that visually indicates to a driver of the vehicle that the interested object is located behind the occluding object when the driver is looking at the electronic display.   
     
     
         9 . The computer program product of  claim 8 , wherein determining, based on the importance value, that the interested object is not allowed to be occluded by the occluding object includes determining that the importance value fails to meet or exceed the importance threshold. 
     
     
         10 . The computer program product of  claim 8 , wherein the non-occluding graphic is an arrow located above the interested object that indicates to the driver a location of the interested object behind the occluding object. 
     
     
         11 . The computer program product of  claim 8 , wherein the sensor data describes a type of object and determining the importance value is based on the type object. 
     
     
         12 . The computer program product of  claim 8 , wherein the computer-readable program further causes the computer to track motion of the interested object based on the sensor data from the external sensors and based on sensor data from one or more vehicle sensors. 
     
     
         13 . The computer program product of  claim 12 , wherein the one or more vehicle sensors includes a camera that tracks the motion of the interested object. 
     
     
         14 . The computer program product of  claim 12 , wherein the vehicle sensors detect the interested object moving from the first position to a second position and the computer-readable program further causes the computer to reposition a location of the non-occluding graphic as displayed on the electronic display to correspond to the second position wherein the non-occluding graphic was previously displayed on the electronic display to correspond to the first position. 
     
     
         15 . A system comprising:
 an electronic display; and   a memory storing instructions that, when executed, cause the system to:
 receive sensor data from external sensors associated with a roadside unit, the sensor data describing a first position of an interested object in an environment external to a vehicle; 
 determine an importance value associated with the interested object; 
 display an initial graphic on the electronic display based on the first position of the interested object; 
 determine that at least a portion of the interested object is occluded by an occluding object; 
 determine that the importance value meets or exceeds an importance threshold; and 
 responsive to a determination that the importance value meets or exceeds the importance threshold, turn off the initial graphic so that the initial graphic is not displayed on the electronic display; and 
   wherein the electronic display displays a non-occluding graphic that does not overlay the occluding object or the interested object and that visually indicates to a driver of the vehicle that the interested object is located behind the occluding object when the driver is looking at the electronic display.   
     
     
         16 . The system of  claim 15 , wherein determining, based on the importance value, that the interested object is not allowed to be occluded by the occluding object includes determining that the importance value fails to meet or exceed the importance threshold. 
     
     
         17 . The system of  claim 15 , wherein the non-occluding graphic is an arrow located above the interested object that indicates to the driver a location of the interested object behind the occluding object. 
     
     
         18 . The system of  claim 15 , wherein the sensor data describes a type of object and determining the importance value is based on the type of object. 
     
     
         19 . The system of  claim 15 , wherein the instructions further cause the system to track motion of the interested object based on the sensor data from the external sensors and based on sensor data from one or more vehicle sensors. 
     
     
         20 . The system of  claim 19 , wherein the one or more vehicle sensors includes a camera that tracks the motion of the interested object.