Vision sensor, image processing device including the same, and operating method of the vision sensor

Abstract

Provided is a vision sensor including a pixel array including a plurality of pixels disposed in a matrix form, an event detection circuit configured to detect whether an event has occurred in the plurality of pixels and generate event signals corresponding to pixels from among the plurality of pixels in which an event has occurred, a map data processor configured to generate a timestamp map based on the event signals, and an interface circuit configured to transmit vision sensor data including at least one of the event signals and the timestamp map to an external processor, wherein the timestamp map includes timestamp information indicating polarity information, address information, and an event occurrence time of a pixel included in an event signal corresponding to the pixel.

What is claimed is: 
     
         1 . A vision sensor comprising:
 a pixel array comprising a plurality of pixels disposed in a matrix form;   an event detection circuit configured to detect whether an event has occurred in the plurality of pixels and generate event signals corresponding to pixels from among the plurality of pixels in which an event has occurred;   a map data processor configured to generate a timestamp map based on the event signals; and   an interface circuit configured to transmit vision sensor data including at least one of the event signals and the timestamp map to an external processor,   wherein the timestamp map includes timestamp information indicating polarity information, address information, and an event occurrence time of a pixel included in an event signal corresponding to the pixel.   
     
     
         2 . The vision sensor of  claim 1 , wherein the event signals are generated in an address-event representation (AER) format including address information, timestamp information, and polarity information of the pixels in which an event has occurred or a raw format including event occurrence information of all of the plurality of pixels. 
     
     
         3 . The vision sensor of  claim 1 , wherein the map data processor is further configured to select event signals generated in a region of interest (ROI) of the pixel array, from among the event signals. 
     
     
         4 . The vision sensor of  claim 1 , wherein, based on a number of events that occurred in a preset N*N size-pixel being greater than a preset threshold value, the map data processor is further configured to determine that an event has occurred in the N*N size-pixel, and store an event signal corresponding to the N*N size-pixel. 
     
     
         5 . The vision sensor of  claim 1 , wherein the interface circuit is further configured to:
 in a first output mode, output an event signal included in the vision sensor data to the external processor;   in a second output mode, generate at least one virtual channel, transmit the event signal to the external processor via a first virtual channel, and transmit the timestamp map data to the external processor via a second virtual channel; and   in a third output mode, generate at least one virtual channel, transmit the event signal to the external processor via the first virtual channel, and transmit optical flow map data to the external processor via the second virtual channel.   
     
     
         6 . The vision sensor of  claim 1 , wherein the map data processor is further configured to receive event signals periodically generated in each of frames, and generate a plurality of timestamp maps based on different frame periods. 
     
     
         7 . The vision sensor of  claim 6 , wherein the map data processor is further configured to periodically generate a first timestamp reference frame and a first timestamp map based on a preset number of event signal frames from the first timestamp reference frame, and periodically generate a second timestamp reference frame and a second timestamp map based on a preset number of event signal frames from the second timestamp reference frame, and
 wherein the first timestamp map and the second timestamp map have overlapping event signal frame periods for generating the first timestamp map and the second timestamp map.   
     
     
         8 . The vision sensor of  claim 1 , wherein the timestamp map is configured to store an offset value of a timestamp based on a preset timestamp reference. 
     
     
         9 . The vision sensor of  claim 8 , wherein the timestamp map is configured to set a timestamp reference based on event occurrence information. 
     
     
         10 . The vision sensor of  claim 1 , wherein the map data processor is further configured to generate an optical flow map by estimating an optical flow including information about a direction and a velocity in and at which a pixel in which an event signal is generated is varied, from the timestamp map, based on a preset algorithm. 
     
     
         11 . The vision sensor of  claim 10 , wherein the map data processor is further configured to store the optical flow map in a location of a map data buffer, in which the timestamp map is stored, instead of the timestamp map. 
     
     
         12 . An image processing device comprising:
 a vision sensor configured to:
 generate a plurality of event signals corresponding to a plurality of pixels in which an event has occurred based on a movement of an object, from among the plurality of pixels included in a pixel array; 
 generate a timestamp map based on polarity information, address information, and event occurrence time information of a pixel included in the plurality of event signals, and output vision sensor data including the event signals and the timestamp map; and 
   a processor configured to detect the movement of the object by processing the vision sensor data output from the vision sensor.   
     
     
         13 . The image processing device of  claim 12 , wherein the vision sensor is further configured to:
 generate an optical flow map by estimating an optical flow including information about a direction and a velocity in and at which a pixel in which an event signal is generated is varied, from the timestamp map based on a preset algorithm; and   replace the generated optical flow map in a location of a map data buffer in which the timestamp map is stored.   
     
     
         14 . The image processing device of  claim 12 , wherein the vision sensor is further configured to generate at least one virtual channel, transmit the event signals to the processor via a first virtual channel, and transmit the timestamp map data or optical flow map data to the processor via a second virtual channel. 
     
     
         15 . The image processing device of  claim 12 , wherein, based on a number of events that occurred in a preset N*N size-pixel being greater than a preset threshold value, the vision sensor determines that an event has occurred. 
     
     
         16 . An operating method of a vision sensor, the operating method comprising:
 detecting whether an event has occurred in a plurality of pixels and generating event signals corresponding to pixels from among the plurality of pixels in which an event has occurred;   generating vision sensor data including a timestamp map based on the event signals; and   transmitting the vision sensor data to an external processor,   wherein the timestamp map includes timestamp information indicating polarity information, address information, and an event occurrence time of a pixel included in an event signal.   
     
     
         17 . The operating method of  claim 16 , wherein the event signals are generated in an address-event representation (AER) format including address information, timestamp information, and polarity information of the pixels in which an event has occurred or a raw format including event occurrence information of all of the plurality of pixels. 
     
     
         18 . The operating method of  claim 16 , further comprising selecting event signals generated in a region of interest (ROI) of a pixel array including the plurality of pixels, from among the event signals. 
     
     
         19 . The operating method of  claim 16 , wherein, based on a number of events that occurred in a preset N*N size-pixel being greater than a preset threshold value, the vision sensor determines that an event has occurred in the preset N*N size-pixel. 
     
     
         20 . The operating method of  claim 16 , further comprising generating an optical flow map by estimating an optical flow including information about a direction and a velocity in and at which a pixel in which an event signal is generated is varied, from the timestamp map based on a preset algorithm.