Methods and systems for selective sensor fusion

Abstract

A method for determining a physical state of a movable object includes determining an estimated physical state based on first sensing data obtained by a first sensing system of the movable object, during a time duration in which second sensing data from a second sensing system is unavailable or is not updated; in response to determining that the second sensing data from the second sensing system becomes available or is updated, determining an observed physical state of the movable object based on the second sensing data; and based on a deviation between the observed physical state and the estimated physical state of the movable object, determining whether to update the physical state of the movable object based on the observed physical state. The first and second sensing systems have different sampling frequencies. The deviation is indicative of a validity of the sensing data of the second sensing system.

What is claimed is: 
     
         1 . A method for determining a physical state of a movable object including a first sensing system and a second sensing system, comprising:
 determining, based on first sensing data obtained by the first sensing system, an estimated physical state of the movable object, during a time duration in which second sensing data from the second sensing system is unavailable or is not updated, the first sensing system and the second sensing system having different sampling frequencies;   in response to determining that the second sensing data from the second sensing system becomes available or is updated, determining an observed physical state of the movable object based on the second sensing data; and   based on a deviation between the observed physical state and the estimated physical state of the movable object, determining whether to update the physical state of the movable object based on the observed physical state, the deviation being indicative of a validity of the sensing data of the second sensing system.   
     
     
         2 . The method of  claim 1 , wherein:
 the first sensing data includes a first set of positional information and a first set of motion information of the movable object; and   the second sensing data includes a second set of positional information and a second set of motion information of the movable object.   
     
     
         3 . The method of  claim 1 , wherein:
 the first sensing system includes an inertial measurement unit (IMU); or   the second sensing system includes a global positioning system (GPS) sensor.   
     
     
         4 . The method of  claim 1 , wherein the second sensing system includes one or more vision sensors. 
     
     
         5 . The method of  claim 1 , further comprising:
 determining the first sensing data based on one or more previously-determined physical states of the movable object.   
     
     
         6 . The method of  claim 1 , wherein determining whether to update the physical state of the movable object based on the observed physical state includes:
 determining, based on the deviation and a predetermined threshold, whether to update the physical state of the movable object based on the observed physical state, including:
 determining to update the physical state of the movable object based on the observed physical state, in response to the deviation being equal to or less than the predetermined threshold; and 
 determining to update the physical state of the movable object without using the observed physical state, in response to the deviation being greater than the predetermined threshold. 
   
     
     
         7 . The method of  claim 6 , further comprising:
 in response to determining to update the physical state of the movable object based on the observed physical state, updating the physical state of the movable object based on the estimated physical state and the observed physical state.   
     
     
         8 . The method of  claim 6 , further comprising:
 in response to determining to update the physical state of the movable object without using the observed physical state, determining the estimated physical stat as the physical state of the movable object.   
     
     
         9 . The method of  claim 6 ,
 wherein:
 the movable object further includes a third sensing system, the sampling frequencies of first sensing system, the second sensing system, and the third sensing system being different from each other; and 
 the observed physical state is a first observed physical state, the deviation is a first deviation, and the predetermined threshold is a first predetermined threshold; 
   the method further comprising:
 obtaining, from sensing data obtained by the third sensing system, a second observed physical state of the movable object, in response to determining that the sensing data of the third sensing system becomes available or is updated; and 
 in response to determining that the second sensing data from the second sensing system becomes available or is updated, based on a second deviation between the first observed physical state and the second observed physical state of the movable object, determining whether to update the physical state of the movable object based on the second observed physical state, the second deviation being indicative of a validity of the sensing data of the third sensing system. 
   
     
     
         10 . The method of  claim 9 , wherein determining whether to update the physical state of the movable object based on the second observed physical state includes:
 determining, based on the second deviation and a second predetermined threshold, whether to update the physical state of the movable object based on the second observed physical state.   
     
     
         11 . The method of  claim 10 , wherein determining, based on the second deviation and the second predetermined threshold, whether to update the physical state of the movable object based on the second observed physical state includes:
 determining to update the physical state of the movable object based on the second observed physical state, in response to the second deviation being equal to or less than the second predetermined threshold; and   determining to update the physical state of the movable object without using the second observed physical state, in response to the second deviation being greater than the second predetermined threshold.   
     
     
         12 . The method of  claim 9 , wherein:
 the first sensing system includes an inertial measurement unit (IMU);   the second sensing system includes a global positioning system (GPS) sensor; and   the third sensing system includes one or more or vision sensors.   
     
     
         13 . The method of  claim 9 , further comprising:
 in response to determining to update the physical state of the movable object based on the second observed physical state:
 updating the physical state of the movable object based on the estimated physical state and the second observed physical state; or 
 updating the physical state of the movable object based on the estimated physical state, the first observed physical state, and the second observed physical state. 
   
     
     
         14 . The method of  claim 13 , further comprising:
 in response to determining to update the physical state of the movable object without using the second observed physical state, updating the physical state of the movable object based on the estimated physical state, the first observed physical state.   
     
     
         15 . A non-transitory computer-readable storage medium with instructions stored thereon that, when executed by a computing system, causes one or more processors to:
 determine, based on first sensing data obtained by a first sensing system of a movable object, an estimated physical state of the movable object, during a time duration in which second sensing data from a second sensing system of the movable object is unavailable or is not updated, the first sensing system and the second sensing system having different sampling frequencies;   in response to determining that the second sensing data from the second sensing system becomes available or is updated, determine an observed physical state of the movable object based on the second sensing data; and   based on a deviation between the observed physical state and the estimated physical state of the movable object, determine whether to update the physical state of the movable object based on the observed physical state, the deviation being indicative of a validity of the sensing data of the second sensing system.   
     
     
         16 . A system for determining a validity of vision sensing by an imaging device of a movable object, comprising one or more processors configured to:
 determine a plurality of observed physical states of the movable object based on image data obtained by a plurality of imaging devices for vision sensing;   determine an estimated physical state of the movable object based on sensing data obtained by an inertial measurement unit (IMU); and   determine a validity of vision sensing by each of the plurality of imaging devices based on:
 a deviation between each of the plurality of observed physical states and one of the plurality of estimated physical states; and 
 a predetermined threshold. 
   
     
     
         17 . The system of  claim 16 , wherein the plurality of imaging devices includes a plurality of first imaging devices mounted on different sides of the movable object and a second imaging device connected to the movable object through a carrier, the second imaging device being rotatable relative to the movable object in one or more directions. 
     
     
         18 . The system of  claim 17 , wherein the one or more processors are further configured to:
 discard image data obtained by the second imaging device.   
     
     
         19 . The system of  claim 16 , wherein the one or more processors are further configured to:
 fuse one of the plurality of observed physical states and the estimated physical state, the one of the plurality of observed physical states corresponding to the estimated physical state and being determined by image data obtained by one of the plurality of first imaging devices.   
     
     
         20 . The system of  claim 16 , wherein:
 the plurality of observed physical states are a plurality of first observed physical states, the deviation is a first deviation, and the predetermined threshold is a first predetermined threshold; and   one or more processors are further configured to:
 obtain a second observed physical state based on sensing date obtained by a global positioning system (GPS) sensor; and 
 determine whether a second deviation between the second observed physical state and the estimated physical state is less than or equal to a second predetermined threshold.