Device, method, and program for detecting position and orientation of target object

Abstract

A device for detecting a position and an orientation of a target object includes a detector, a selector, and a position and orientation determination module. The detector detects a feature region from a two-dimensional image obtained by imaging a target object, the feature region being included in a surface element constituting a surface of the target object. The selector selects data of three or more points from three-dimensional point cloud data obtained by three-dimensionally measuring the target object, the data of the three or more points being associated with the feature region. The position and orientation determination module determines position and orientation information by using the data of the three or more points and prior information regarding a shape of the surface element, the position and orientation information being related to a position and an orientation of the surface element.

1 . A device for detecting a position and an orientation of a target object, the device comprising:
 a detection unit configured to detect a feature region from a two-dimensional image obtained by imaging a target object, the feature region being included in a surface element constituting a surface of the target object;   a selection unit configured to select data of at least three points from three-dimensional point cloud data obtained by three-dimensionally measuring the target object, the data of the at least three points being associated with the feature region; and   a position and orientation determination unit configured to determine position and orientation information by using the data of the at least three points and prior information regarding a shape of the surface element, the position and orientation information being related to a position and an orientation of the surface element.   
     
     
         2 . The device according to  claim 1 , wherein the data of the at least three points includes data of at least one first point, and a relative position of the at least one first point with respect to the feature region satisfies a predetermined condition. 
     
     
         3 . The device according to  claim 2 , wherein the predetermined condition is a condition that defines a distance between a point obtained by projecting the at least one first point on the feature region and a representative point representing a position of the feature region. 
     
     
         4 . The device according to  claim 3 , wherein the predetermined condition is a condition that defines an angle between a straight line passing through the point obtained by projecting the at least one first point on the feature region and the representative point, and a representative direction representing an orientation of the feature region. 
     
     
         5 . The device according to  claim 2 , wherein
 the data of the at least three points includes data of at least one second point in addition to the data of the at least one first point, and   a distance between the at least one second point and the at least one first point is within a defined range.   
     
     
         6 . The device according to  claim 1 , wherein the data of the at least three points is data of points belonging to the feature region when each of the points expressed by the three-dimensional point cloud data is projected on the two-dimensional image. 
     
     
         7 . The device according to  claim 1 , wherein the detection unit detects the feature region using at least one of a feature portion existing in the surface element and a contour of the surface element in the two-dimensional image. 
     
     
         8 . The device according to  claim 1 , wherein the surface element is a flat surface or a curved surface having a constant curvature. 
     
     
         9 . The device according to  claim 1 , further comprising:
 an imaging unit configured to image the target object; and   an irradiation unit configured to irradiate the target object with pattern light,   wherein the two-dimensional image is obtained by imaging by the imaging unit in a state where the pattern light is not emitted, and   the three-dimensional point cloud data is generated from an image obtained by imaging by the imaging unit in a state where the pattern light is emitted.   
     
     
         10 . The device according to  claim 1 , wherein the position and orientation information indicates a normal vector at one point in the surface element. 
     
     
         11 . The device according to  claim 10 , further comprising:
 an action instruction generation unit configured to generate an action instruction based on the normal vector, the action instruction being for a robot that picks the target object.   
     
     
         12 . A method for detecting a position and an orientation of a target object, the method comprising:
 detecting a feature region from a two-dimensional image obtained by imaging a target object, the feature region being included in a surface element constituting a surface of the target object;   selecting data of at least three points from three-dimensional point cloud data obtained by three-dimensionally measuring the target object, the data of the at least three points being associated with the feature region; and   determining position and orientation information by using the data of the at least three points and prior information regarding a shape of the surface element, the position and orientation information being related to a position and an orientation of the surface element.   
     
     
         13 . A non-transitory computer-readable storage medium storing a computer program which, when executed by a computer, causes the computer to perform operations comprising
 detecting a feature region from a two-dimensional image obtained by imaging a target object, the feature region being included in a surface element constituting a surface of the target object;   selecting data of at least three points from three-dimensional point cloud data obtained by three-dimensionally measuring the target object, the data of the at least three points being associated with the feature region; and   determining position and orientation information by using the data of the at least three points and prior information regarding a shape of the surface element, the position and orientation information being related to a position and an orientation of the surface element.   
     
     
         14 . The method according to  claim 12 , wherein the data of the at least three points includes data of at least one first point, and a relative position of the at least one first point with respect to the feature region satisfies a predetermined condition. 
     
     
         15 . The method according to  claim 14 , wherein the predetermined condition is a condition that defines a distance between a point obtained by projecting the at least one first point on the feature region and a representative point representing a position of the feature region. 
     
     
         16 . The method according to  claim 12 , wherein the data of the at least three points is data of points belonging to the feature region when each of the points expressed by the three-dimensional point cloud data is projected on the two-dimensional image. 
     
     
         17 . The method according to  claim 12 , wherein detecting the feature region includes detecting the feature region using at least one of a feature portion existing in the surface element and a contour of the surface element in the two-dimensional image. 
     
     
         18 . The method according to  claim 12 , wherein the surface element is a flat surface or a curved surface having a constant curvature. 
     
     
         19 . The method according to  claim 12 , wherein the position and orientation information indicates a normal vector at one point in the surface element. 
     
     
         20 . The method according to  claim 19 , further comprising:
 generating an action instruction based on the normal vector, the action instruction being for a robot that picks the target object.