Posture estimation method and robot

Abstract

An image recognition method according to one aspect of the present invention acquires a camera image generated by capturing a subject using a camera (three-dimensional sensor). A plurality of coordinates corresponding to a plurality of pixels included in a predetermined area in the camera image are acquired. Subject distance information indicating a distance from the subject to the camera in the plurality of pixels is acquired. Then the posture of the subject surface included in the subject in the predetermined area is estimated based on the plurality of coordinates and the plurality of pieces of subject distance information that have been acquired.

1 : A posture estimation method comprising:
 acquiring a captured image generated by capturing a subject using an imaging apparatus;   acquiring a plurality of coordinates corresponding to a plurality of pixels included in a predetermined area in the captured image;   acquiring subject distance information indicating a distance from the subject to the imaging apparatus in the plurality of pixels; and   estimating a posture of the subject surface included in the subject in the predetermined area based on the plurality of pieces of subject distance information and the plurality of coordinates that have been acquired.   
     
     
         2 : The posture estimation method according to  claim 1 , comprising:
 acquiring a distance image where each pixel includes the subject distance information;   associating the pixels in the captured image with the pixels in the distance image; and   acquiring the subject distance information from pixels corresponding to the plurality of pixels in the predetermined area of the pixels in the distance image.   
     
     
         3 : The posture estimation method according to  claim 1 , comprising:
 calculating three-dimensional coordinates of the plurality of pixels based on the subject distance information and coordinates of the plurality of pixels; and   estimating the posture of the subject surface included in the predetermined area based on the three-dimensional coordinates of the plurality of pixels.   
     
     
         4 : The posture estimation method according to  claim 1 , wherein
 a marker is attached to the subject surface, and   the posture estimation method further comprises:   detecting a marker area including the marker in the captured image as the predetermined area; and   estimating the posture of the marker included in the marker area that has been detected.   
     
     
         5 : The posture estimation method according to  claim 4 , comprising:
 calculating an equation of a projection plane that is parallel to the subject surface using the subject distance information and the coordinates of the plurality of pixels;   projecting a feature point indicating the posture of the marker in the captured image onto the projection plane; and   estimating the posture of the marker based on the coordinates of the feature point projected onto the projection plane.   
     
     
         6 : The posture estimation method according to  claim 5 ,
 comprising specifying the coordinates of the feature point in the captured image with sub-pixel precision to project the feature point onto the projection plane using the coordinates of the feature point that have been specified.   
     
     
         7 : The posture estimation method according to  claim 5 , comprising:
 estimating the position of the marker based on the coordinates of the feature point projected onto the projection plane;   calculating the coordinates of the feature point on the projection plane using information regarding the estimated posture of the marker, the estimated position of the marker, and the size of the marker that has been set in advance;   projecting the feature point that has been calculated on the projection plane onto the captured image;   comparing, in the captured image, the coordinates of the feature point when the subject is captured with the coordinates of the feature point that have been projected; and   determining an estimation accuracy based on the comparison result.   
     
     
         8 : The posture estimation method according to  claim 5 , comprising:
 estimating the position of the marker based on the coordinates of the feature point projected onto the projection plane;   calculating the coordinates of the feature point on the projection plane using information regarding the estimated posture of the marker, the estimated position of the marker, and the size of the marker that has been set in advance;   comparing the coordinates of the feature point projected from the captured image when the posture of the marker is estimated with the coordinates of the feature point that have been calculated on the projection plane; and   determining an estimation accuracy based on the comparison result.   
     
     
         9 : The posture estimation method according to  claim 5 , wherein
 the marker has a substantially rectangular shape, and   the posture estimation method further comprises:   detecting the apices of the marker in the captured image as the feature points;   extending sides of the marker extending from the feature points that have been detected when the number of feature points that have been detected is two or three; and   estimating the intersection of the sides that have been extended as the feature point.   
     
     
         10 : The posture estimation method according to  claim 5 , wherein
 the marker has a substantially rectangular shape, and   the posture estimation method further comprises:   detecting the apices of the marker in the captured image as the feature points;   extending sides of the marker extending from the feature points that have been detected when the number of feature points that have been detected is four or less; and   estimating a point that is located on the sides that have been extended and is spaced apart from the feature points that have been detected by a predetermined distance as the feature point.   
     
     
         11 : A robot comprising:
 the imaging apparatus;   a distance sensor that acquires the subject distance information; and   a posture estimation device that executes the posture estimation method according to  claim 1 .