Robot control device, robot control method, and non-transitory computer readable storage medium

Abstract

A robot control device is provided to accept information ( 201 ) specifying an object 170 ) manipulated by a robot ( 20 ) from among objects of a plurality of kinds and accept information ( 202 ) specifying a target relative positional relationship between the specified object ( 70 ) and the distal end of a hand of the robot ( 20 ). The robot control device extracts the object ( 70 ) from image information ( 501 ) obtained by photographing the objects of the plurality of kinds and the surrounding environment thereof, generates information ( 301 ) indicating the position and orientation of the object ( 70 ), generates an action instruction ( 401 ) from the result of learning by a learning module ( 103 ), the action instruction ( 401 ) serving to match the relative positional relationship between the object ( 70 ) and the distal end of the hand of the robot ( 20 ) with the target relative positional relationship, and outputs the action instruction ( 401 ) to the robot ( 20 ).

1 . A robot control device, comprising:
 an input unit configured to receive an input of information that specifies a specific kind of object to be manipulated by a robot among a plurality of kinds of objects and an input of information that specifies a target relative positional relationship between the specific kind of object specified and a distal end of a hand of the robot;   an imaging device configured to image the plurality of kinds of objects and the surrounding environment;   an extraction unit configured to extract the specific kind of object specified from image information obtained by imaging by the imaging device and generate information indicating the position and orientation of the specific kind of object;   a detection device configured to detect the position and orientation of the robot;   a learning module including a trained model in which the action of the robot has undergone machine learning so that the relative positional relationship between the specific kind of object specified and the distal end of the hand matches the target relative positional relationship or a model having the same input and output relationship as the trained model; and   an output unit configured to generate an action instruction that instructs the robot to perform an action so that the relative positional relationship between the specific kind of object and the distal end of the hand determined based on the information indicating the position and orientation of the specific kind of object generated by the extraction unit and information indicating the position and orientation of the robot detected by the detection device matches the target relative positional relationship from the learning result of the learning module and output the instruction to the robot.   
     
     
         2 . The robot control device according to  claim 1 ,
 wherein the target relative positional relationship between the specific kind of object and the distal end of the hand is constant regardless of the positions and orientations of the robot and the specific kind of object.   
     
     
         3 . The robot control device according to  claim 1 ,
 wherein the target relative positional relationship between the specific kind of object and the distal end of the hand is able to be changed by specification of an operator.   
     
     
         4 . The robot control device according to  claim 1 ,
 wherein the input unit includes a graphical user interface for specifying the target relative positional relationship between the specific kind of object and the distal end of the hand.   
     
     
         5 . The robot control device according to  claim 1 ,
 wherein information that specifies the specific kind of object is input to the extraction unit.   
     
     
         6 . A robot control method causing a computer system to execute:
 a step of receiving an input of information that specifies a specific kind of object to be manipulated by a robot among a plurality of kinds of objects;   a step of receiving an input of information that specifies a target relative positional relationship between the specific kind of object specified and a distal end of a hand of the robot;   a step of imaging the plurality of kinds of objects and the surrounding environment;   a step of extracting the specific kind of object specified from image information obtained by the imaging and generating information indicating the position and orientation of the specific kind of object;   a step of detecting the position and orientation of the robot;   a step of preparing a learning module including a trained model in which the action of the robot has undergone machine learning so that the relative positional relationship between the specific kind of object specified and the distal end of the hand matches the target relative positional relationship or a model having the same input and output relationship as the trained model; and   a step of generating an action instruction that instructs the robot to perform an action so that the relative positional relationship between the specific kind of object and the distal end of the hand determined based on the information indicating the position and orientation of the specific kind of object and information indicating the position and orientation of the robot matches the target relative positional relationship from the learning result of the learning module and outputting the instruction to the robot.   
     
     
         7 . A non-transitory computer readable storage medium storing robot control program causing a computer system to execute:
 a step of receiving an input of information that specifies a specific kind of object to be manipulated by a robot among a plurality of kinds of objects;   a step of receiving an input of information that specifies a target relative positional relationship between the specific kind of object specified and a distal end of a hand of the robot;   a step of imaging the plurality of kinds of objects and the surrounding environment;   a step of extracting the specific kind of object specified from image information obtained by the imaging and generating information indicating the position and orientation of the specific kind of object;   a step of detecting the position and orientation of the robot;   a step of preparing a learning module including a trained model in which the action of the robot has undergone machine learning so that the relative positional relationship between the specific kind of object specified and the distal end of the hand matches the target relative positional relationship or a model having the same input and output relationship as the trained model; and   a step of generating an action instruction that instructs the robot to perform an action so that the relative positional relationship between the specific kind of object and the distal end of the hand determined based on the information indicating the position and orientation of the specific kind of object and information indicating the position and orientation of the robot matches the target relative positional relationship from the learning result of the learning module and outputting the instruction to the robot.   
     
     
         8 . The robot control device according to  claim 2 ,
 wherein the target relative positional relationship between the specific kind of object and the distal end of the hand is able to be changed by specification of an operator.   
     
     
         9 . The robot control device according to  claim 2 ,
 wherein the input unit includes a graphical user interface for specifying the target relative positional relationship between the specific kind of object and the distal end of the hand.   
     
     
         10 . The robot control device according to  claim 3 ,
 wherein the input unit includes a graphical user interface for specifying the target relative positional relationship between the specific kind of object and the distal end of the hand.   
     
     
         11 . The robot control device according to  claim 2 ,
 wherein information that specifies the specific kind of object is input to the extraction unit.   
     
     
         12 . The robot control device according to  claim 3 ,
 wherein information that specifies the specific kind of object is input to the extraction unit.   
     
     
         13 . The robot control device according to  claim 4 ,
 wherein information that specifies the specific kind of object is input to the extraction unit.