Synchronizing operations of lidar and tof sensors

Abstract

Systems and techniques are provided for synchronizing sensor operations. An example method includes determining a scanning frequency of a light detection and ranging (LIDAR) sensor configured to collect data for regions of space during each scan cycle; selecting an exposure from an exposure sequence generated based on data captured by a time-of-flight (TOF) sensor to align with data from a scan from the LIDAR sensor during a scan cycle; based on the scanning frequency, a field-of-view (FOV) of the LIDAR sensor, a FOV of the TOF sensor, a location of the exposure within the exposure sequence, and/or sensor internal delays, determining a timeframe between a reference time and an alignment time during the scan cycle when the FOVs of the LIDAR sensor and the TOF sensor are aligned; and based on the timeframe, determining a time offset for triggering the TOF sensor to capture data associated with the exposure sequence.

What is claimed is: 
     
         1 . A system comprising:
 a memory; and   one or more processors coupled to the memory, the one or more processors being configured to:
 determine a frequency of each scan cycle of a light detection and ranging (LIDAR) sensor configured to collect sensor data for different regions of space as the LIDAR sensor scans in different directions during each scan cycle; 
 select an exposure from an exposure sequence generated based on data captured by a time-of-flight (TOF) sensor to align with LIDAR data from a LIDAR scan associated with the sensor data collected by the LIDAR sensor during a scan cycle; 
 based on the frequency of the scan cycle of the LIDAR sensor, a field-of-view (FOV) of the LIDAR sensor, a FOV of the TOF sensor, and a location of the exposure within the exposure sequence, determine an amount of time estimated to lapse between a reference time and an alignment time during the scan cycle when a first point within the FOV of the LIDAR sensor is aligned in space with a second point within the FOV of the TOF sensor; 
 based on the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle, determine a time offset for triggering the TOF sensor to capture data associated with the exposure sequence; and 
 send, to the TOF sensor, a signal configured to trigger the TOF sensor to capture the data associated with the exposure sequence at a time associated with the time offset. 
   
     
     
         2 . The system of  claim 1 , wherein the one or more processors are configured to:
 determine a time delay between a time when the TOF sensor initiates an operation to capture the data and a different time when the TOF sensor captures the data, wherein the time offset comprises the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle minus the time delay.   
     
     
         3 . The system of  claim 1 , wherein the time offset is configured to trigger the TOF sensor to capture data associated with the exposure for a portion of an exposure time associated with the exposure at or by the alignment time during the scan cycle when the first point within the FOV of the LIDAR sensor is aligned in space with the second point within the FOV of the TOF sensor. 
     
     
         4 . The system of  claim 3 , wherein the portion of the exposure time comprises half of the exposure time. 
     
     
         5 . The system of  claim 3 , wherein the portion of the exposure time comprises less or more than half of the exposure time. 
     
     
         6 . The system of  claim 1 , wherein the time offset is configured to trigger the TOF sensor to start capturing data associated with the exposure at or by the alignment time during the scan cycle when the first point within the FOV of the LIDAR sensor is aligned in space with the second point within the FOV of the TOF sensor. 
     
     
         7 . The system of  claim 1 , wherein the reference time comprises a beginning of the scan cycle, a beginning of a previous scan cycle associated with the LIDAR sensor, a time within the previous scan cycle, a time from a reference clock, a time when the TOF sensor was triggered to capture data during one or more previous scan cycles associated with the LIDAR sensor, and a time associated with a data capture operation performed by the TOF sensor during the previous scan cycle. 
     
     
         8 . The system of  claim 1 , wherein the exposure is located sequentially after one or more exposures from the exposure sequence, and wherein determining the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle comprises:
 determining an exposure time of each of the one or more exposures; and   determining a time delay between a time when the TOF sensor initiates an operation to capture the data and a different time when the TOF sensor captures the data, wherein the time offset comprises the exposure time and the time delay.   
     
     
         9 . The system of  claim 1 , wherein the first point within the FOV of the LIDAR sensor is on a first plane that intersects a center of the FOV of the LIDAR sensor or extends from a vertex of a first angle of the FOV of the LIDAR sensor, and wherein the second point within the FOV of the TOF sensor is on a second plane that intersects a center of the FOV of the TOF sensor or extends from a vertex of a second angle of the FOV of the TOF sensor. 
     
     
         10 . A method comprising:
 determining a frequency of each scan cycle of a light detection and ranging (LIDAR) sensor configured to collect sensor data for different regions of space as the LIDAR sensor scans in different directions during each scan cycle;   selecting an exposure from an exposure sequence generated based on data captured by a time-of-flight (TOF) sensor to align with LIDAR data from a LIDAR scan associated with the sensor data collected by the LIDAR sensor during a scan cycle;   based on the frequency of the scan cycle of the LIDAR sensor, a field-of-view (FOV) of the LIDAR sensor, a FOV of the TOF sensor, and a location of the exposure within the exposure sequence, determining an amount of time estimated to lapse between a reference time and an alignment time during the scan cycle when a first point within the FOV of the LIDAR sensor is aligned in space with a second point within the FOV of the TOF sensor;   based on the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle, determining a time offset for triggering the TOF sensor to capture data associated with the exposure sequence; and   sending, to the TOF sensor, a signal configured to trigger the TOF sensor to capture the data associated with the exposure sequence at a time associated with the time offset.   
     
     
         11 . The method of  claim 10 , further comprising:
 determining a time delay between a time when the TOF sensor initiates an operation to capture the data and a different time when the TOF sensor captures the data, wherein the time offset comprises the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle minus the time delay.   
     
     
         12 . The method of  claim 10 , wherein the time offset is configured to trigger the TOF sensor to capture data associated with the exposure for a portion of an exposure time associated with the exposure at or by the alignment time during the scan cycle when the first point within the FOV of the LIDAR sensor is aligned in space with the second point within the FOV of the TOF sensor. 
     
     
         13 . The method of  claim 12 , wherein the portion of the exposure time comprises half of the exposure time. 
     
     
         14 . The method of  claim 12 , wherein the portion of the exposure time comprises less or more than half of the exposure time. 
     
     
         15 . The method of  claim 10 , wherein the time offset is configured to trigger the TOF sensor to start capturing data associated with the exposure at or by the alignment time during the scan cycle when the first point within the FOV of the LIDAR sensor is aligned in space with the second point within the FOV of the TOF sensor. 
     
     
         16 . The method of  claim 10 , wherein the reference time comprises a beginning of the scan cycle, a beginning of a previous scan cycle associated with the LIDAR sensor, a time within the previous scan cycle, a time from a reference clock, a time when the TOF sensor was triggered to capture data during one or more previous scan cycles associated with the LIDAR sensor, and a time associated with a data capture operation performed by the TOF sensor during the previous scan cycle. 
     
     
         17 . The method of  claim 10 , wherein the exposure is located sequentially after one or more exposures from the exposure sequence, and wherein determining the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle comprises:
 determining an exposure time of each of the one or more exposures; and   determining a time delay between a time when the TOF sensor initiates an operation to capture the data and a different time when the TOF sensor captures the data, wherein the time offset comprises the exposure time and the time delay.   
     
     
         18 . The method of  claim 10 , wherein the first point within the FOV of the LIDAR sensor is on a first plane that intersects a center of the FOV of the LIDAR sensor or extends from a vertex of a first angle of the FOV of the LIDAR sensor, and wherein the second point within the FOV of the TOF sensor is on a second plane that intersects a center of the FOV of the TOF sensor or extends from a vertex of a second angle of the FOV of the TOF sensor. 
     
     
         19 . The method of  claim 10 , wherein the LIDAR sensor and the TOF sensor are mounted on a vehicle. 
     
     
         20 . A non-transitory computer-readable medium having stored thereon instructions which, when executed by one or more processors, cause the one or more processors to:
 determine a frequency of each scan cycle of a light detection and ranging (LIDAR) sensor configured to collect sensor data for different regions of space as the LIDAR sensor scans in different directions during each scan cycle;   select an exposure from an exposure sequence generated based on data captured by a time-of-flight (TOF) sensor to align with LIDAR data from a LIDAR scan associated with the sensor data collected by the LIDAR sensor during a scan cycle;   based on the frequency of the scan cycle of the LIDAR sensor, a field-of-view (FOV) of the LIDAR sensor, a FOV of the TOF sensor, and a location of the exposure within the exposure sequence, determine an amount of time estimated to lapse between a reference time and an alignment time during the scan cycle when a first point within the FOV of the LIDAR sensor is aligned in space with a second point within the FOV of the TOF sensor;   based on the amount of time estimated to lapse between the reference time and the alignment time during the scan cycle, determine a time offset for triggering the TOF sensor to capture data associated with the exposure sequence; and   send, to the TOF sensor, a signal configured to trigger the TOF sensor to capture the data associated with the exposure sequence at a time associated with the time offset.