Sensor Fusion to Determine Reliability of Autonomous Vehicle Operation

Abstract

A method for an autonomous vehicle includes: receiving first object data from a first sensor module; receiving second object data from a second sensor module; comparing the first object data to the second object data; determining, based on comparing the first object data to the second object data, whether the first object data corresponds to the second object data; and in response to determining that the first object data does not correspond to the second object data, performing an action for the autonomous vehicle.

What is claimed is: 
     
         1 . A system comprising:
 a central processing device configured to receive object data provided by sensors of a vehicle; and   memory storing instructions configured to instruct the central processing device to:
 receive first object data based on a first object detection by a first sensor of the vehicle; 
 receive second object data based on a second object detection by a second sensor of the vehicle; 
 make a comparison of the first object data to the second object data, the comparison comprising performing a correlation of the first object data to the second object data; 
 determine, based on the comparison of the first object data to the second object data, whether the first object data corresponds to the second object data; and 
 in response to determining that the first object data does not correspond to the second object data, change a configuration of at least one of the first sensor or the second sensor. 
   
     
     
         2 . The system of  claim 1 , wherein the instructions are further configured to instruct the central processing device to:
 in response to determining that the first object data does not correspond to the second object data, perform a diagnostic test of at least one of the first sensor or the second sensor;   wherein the configuration of the first sensor or the second sensor is changed based on the diagnostic test.   
     
     
         3 . The system of  claim 2 , wherein the instructions are further configured to instruct the central processing device to determine, based on the diagnostic test, that a malfunction exists in at least one of the first sensor or the second sensor. 
     
     
         4 . The system of  claim 1 , wherein the instructions are further configured to instruct the central processing device to, in response to determining that the first object data does not correspond to the second object data, disable an autonomous mode of operation for the vehicle. 
     
     
         5 . The system of  claim 1 , wherein the correlation of the first object data to the second object data provides a result, and the instructions are further configured to instruct the central processing device to:
 determine an expected correlation; and   compare the result to the expected correlation.   
     
     
         6 . The system of  claim 5 , wherein:
 the instructions are further configured to instruct the central processing device to determine, based on at least one of the first object data or the second object data, a type of object; and   the expected correlation is determined based on the type of object.   
     
     
         7 . The system of  claim 1 , wherein determining whether the first object data corresponds to the second object data comprises comparing a number of times that the first object data matches the second object data to a threshold. 
     
     
         8 . The system of  claim 7 , wherein the threshold varies based on a time of day. 
     
     
         9 . A vehicle comprising:
 a first sensor configured to store a first object type;   a second sensor configured to store a second object type; and   a central processing device configured to:
 receive first object data from the first sensor; 
 receive second object data from the second sensor; 
 make a comparison, based on the first object type or the second object type, of the first object data to the second object data; 
 determine, based on the comparison of the first object data to the second object data, whether the first object data corresponds to the second object data; and 
 in response to determining that the first object data does not correspond to the second object data, change a configuration of at least one of the first sensor or the second sensor. 
   
     
     
         10 . The vehicle of  claim 9 , wherein making the comparison of the first object data to the second object data comprises comparing the first object type to the second object type. 
     
     
         11 . The vehicle of  claim 9 , wherein the comparison of the first object data to the second object data comprises performing a correlation of the first object data to the second object data. 
     
     
         12 . The vehicle of  claim 11 , wherein the correlation is based on a type of object. 
     
     
         13 . The vehicle of  claim 9 , wherein the first object data comprises a position of an object detected by the first sensor, and the second object data comprises a position of an object detected by the second sensor. 
     
     
         14 . The vehicle of  claim 9 , wherein the central processing device is further configured to:
 determine a context of the vehicle based on data from at least one sensor other than the first sensor and the second sensor;   wherein determining whether the first object data corresponds to the second object data is based in part on the context.   
     
     
         15 . The vehicle of  claim 9 , wherein the first sensor includes memory storing data for a neural network, and wherein the first sensor uses the neural network to evaluate data collected by the first sensor. 
     
     
         16 . The vehicle of  claim 9 , wherein:
 the determining comprises comparing a number of times that the first object data matches the second object data to a threshold; and   the threshold is a predetermined percentage of a total number of comparisons.   
     
     
         17 . A vehicle comprising:
 a lidar sensor;   a camera; and   a central processing device configured to:
 receive first object data from the lidar sensor; 
 receive second object data from the camera; 
 make a comparison of the first object data to the second object data; 
 determine, based on the comparison of the first object data to the second object data, whether the first object data corresponds to the second object data; and 
 in response to determining that the first object data does not correspond to the second object data, change a configuration of at least one of the lidar sensor or the camera. 
   
     
     
         18 . The vehicle of  claim 17 , wherein determining whether the first object data corresponds to the second object data comprises determining whether a statistical detection relationship is maintained between the first object data and the second object data. 
     
     
         19 . The vehicle of  claim 17 , wherein the central processing device is further configured to, in response to determining that the first object data does not correspond to the second object data, require a person to take over control of the vehicle. 
     
     
         20 . The vehicle of  claim 17 , wherein the central processing device is further configured to:
 perform a correlation of the first object data to the second object data to provide a result;   determine an expected correlation; and   compare the result to the expected correlation.   
     
     
         21 . The vehicle of  claim 20 , wherein:
 the central processing device is further configured to determine, based on at least one of the first object data or the second object data, a type of object; and   the expected correlation is determined based on the determined type of object.