System and method of object detection based on image data

Abstract

A system and method of detecting objects are provided. The method includes generating first edge information from first image data representing an object based on a first mode of image capture, generating second edge information from second image data representing the object based on a second mode of image capture, the second mode being different from the first mode, fusing the first edge information with the second edge information to generate fused edge information, generating an object detection hypothesis based on the fused edge information, and validating the object detection hypothesis based on the fused edge information, the first edge information, and/or the second edge information.

I claim: 
     
         1 . A method of detecting objects, the method comprising:
 generating first edge information from first image data representing an object based on a first mode of image capture;   generating second edge information from second image data representing the object based on a second mode of image capture, the second mode being different from the first mode;   fusing the first edge information with the second edge information to generate fused edge information;   generating an object detection hypothesis based on the fused edge information; and   validating the object detection hypothesis based on the fused edge information, the first edge information, and/or the second edge information.   
     
     
         2 . The method of  claim 1 , wherein fusing the first edge information with the second edge information includes combining the first edge information with the second edge information into a common image space. 
     
     
         3 . The method of  claim 2 , wherein combining the first edge information with the second edge information into the common image space includes:
 mapping the first edge information based on coordinates of detected edges associated with the first mode of image capture; and   transforming the second edge information to correspond to an image space associated with the first mode of image capture.   
     
     
         4 . The method of  claim 3 , wherein the image space associated with the first mode of image capture is a three-dimensional image space; and
 wherein transforming the second edge information to correspond to the image space associated with the first mode of image capture includes:   projecting two-dimensional edge points from the second edge information onto the three-dimensional image space based on depth information associated with the first edge information corresponding to the two-dimensional locations of the two-dimensional edge points of the second edge information to generate projected three-dimensional points associated with the second edge information; and   transforming the projected three-dimensional points to correspond to the three-dimensional image space.   
     
     
         5 . The method of  claim 4 , wherein transforming the second edge information to correspond to the image space associated with the first mode of image capture further comprises:
 projecting the transformed three-dimensional points corresponding to the three-dimensional image space to two-dimensional coordinates of images associated with the first edge information.   
     
     
         6 . The method of  claim 1 , wherein the image data representing the object based on the first mode of image capture is image data captured by a mode of two-dimensional image capture, and
 wherein detecting the first edge information includes one or more of:   CANNY Edge Detection;   Region Similarity Edge Detection; and   Line Segment Detection.   
     
     
         7 . The method of  claim 1 , wherein the image data representing the object based on the first mode of image capture is image data captured by a mode of three-dimensional image capture, and
 wherein detecting the first edge information includes:   projecting the image data captured using the first mode into an image space associated with one or more objects to be detected; and   extracting object edges from a projected 3D point cloud.   
     
     
         8 . A non-transitory computer readable medium encoded with instructions for making a computing device execute a method of detecting objects, the method comprising:
 generating first edge information from first image data representing an object based on a first mode of image capture;   generating second edge information from second image data representing the object based on a second mode of image capture, the second mode being different from the first mode;   fusing the first edge information with the second edge information to generate fused edge information;   generating an object detection hypothesis based on the fused edge information; and   validating the object detection hypothesis based on the fused edge information, the first edge information, and/or the second edge information.   
     
     
         9 . The non-transitory computer readable medium of  claim 8 , wherein fusing the first edge information with the second edge information includes combining the first edge information with the second edge information into a common image space. 
     
     
         10 . The non-transitory computer readable medium of  claim 9 , wherein combining the first edge information with the second edge information into the common image space includes:
 mapping the first edge information based on coordinates of detected edges associated with the first mode of image capture; and   transforming the second edge information to correspond to an image space associated with the first mode of image capture.   
     
     
         11 . The non-transitory computer readable medium of  claim 10 , wherein the image space associated with the first mode of image capture is a three-dimensional image space; and
 wherein transforming the second edge information to correspond to the image space associated with the first mode of image capture includes:   projecting two-dimensional edge points from the second edge information onto the three-dimensional image space based on depth information associated with the first edge information corresponding to the two-dimensional locations of the two-dimensional edge points of the second edge information to generate projected three-dimensional points associated with the second edge information; and   transforming the projected three-dimensional points to correspond to the three-dimensional image space.   
     
     
         12 . The non-transitory computer readable medium of  claim 11 , wherein transforming the second edge information to correspond to the image space associated with the first mode of image capture further comprises:
 projecting the transformed three-dimensional points corresponding to the three-dimensional image space to two-dimensional coordinates of images associated with the first edge information.   
     
     
         13 . The non-transitory computer readable medium of  claim 8 , wherein the first image data representing the object based on the first mode of image capture is image data captured by a mode of two-dimensional image capture, and
 wherein detecting the first edge information includes one or more of:   CANNY Edge Detection;   Region Similarity Edge Detection; and   Line Segment Detection.   
     
     
         14 . The non-transitory computer readable medium of  claim 8 , wherein the first image data representing the object based on the first mode of image capture is image data captured by a mode of three-dimensional image capture, and
 wherein detecting the first edge information includes:   projecting the image data captured using the first mode into an image space associated with one or more objects to be detected; and   extracting object edges from a projected 3D point cloud.   
     
     
         15 . An object detection system for detecting objects, the system comprising:
 a system interface configured for receiving first image data representing an object based on a first mode of image capture and receiving second image data representing the object based on a second mode of image capture, the second mode being different from the first mode;   a processor, communicatively coupled to the system interface, the processor configured to perform an object detection method comprising:
 generating first edge information from the first image data representing an object based on a first mode of image capture; 
 generating second edge information from the second image data representing an object based on a second mode of image capture, the second mode being different from the first mode; 
 fusing the first edge information with the second edge information to generate fused edge information; 
 generating an object detection hypothesis based on the fused edge information; and 
 validating the object detection hypothesis based on the fused edge information, the first edge information, and/or the second edge information. 
   
     
     
         16 . The object detection system of  claim 15 , wherein fusing the first edge information with the second edge information includes combining the first edge information with the second edge information into a common image space. 
     
     
         17 . The object detection system of  claim 16 , wherein combining the first edge information with the second edge information into the common image space includes:
 mapping the first edge information based on coordinates of detected edges associated with the first mode of image capture; and   transforming the second edge information to correspond to an image space associated with the first mode of image capture.   
     
     
         18 . The object detection system of  claim 17 , wherein the image space associated with the first mode of image capture is a three-dimensional image space; and
 wherein transforming the second edge information to correspond to the image space associated with the first mode of image capture includes:   projecting two-dimensional edge points from the second edge information onto the three-dimensional image space based on depth information associated with the first edge information corresponding to the two-dimensional locations of the two-dimensional edge points of the second edge information to generate projected three-dimensional points associated with the second edge information; and   transforming the projected three-dimensional points to correspond to the three-dimensional image space.   
     
     
         19 . The object detection system of  claim 18 , wherein transforming the second edge information to correspond to the image space associated with the first mode of image capture further comprises:
 projecting the transformed three-dimensional points corresponding to the three-dimensional image space to two-dimensional coordinates of images associated with the first edge information.   
     
     
         20 . The object detection system of  claim 15 , wherein the first image data representing the object based on the first mode of image capture is image data captured by a mode of three-dimensional image capture, and
 wherein detecting the first edge information includes:   projecting the image data captured using the first mode into an image space associated with one or more objects to be detected; and   extracting object edges from a projected 3D point cloud.