Grasp Planning Of Unknown Object For Digital Human Model

Abstract

An embodiment receives models of an object and an environment and an indication of position of a digital human model (DHM). An oriented bounding box (with a plurality of faces) surrounding the model of the object is determined and, for each of the plurality of faces, a candidate grasp location, a candidate grasp orientation, and a candidate grasp type is determined. From amongst the plurality of faces, one or more graspable faces is determined based on: the candidate grasp locations, the candidate grasp orientations, the environment model, and dimensions of each face. Then, an optimal graspable face is identified based on a hierarchy and the position of the DHM. An inverse kinematic solver determines position and orientation, i.e., grasp, of an end effector of the DHM grasping the object based on the candidate grasp location, candidate grasp orientation, and candidate grasp type of the optimal graspable face.

What is claimed is: 
     
         1 . A computer-implemented method of determining position and orientation of an end effector of a digital human model (DHM) for grasping an object, the method comprising:
 receiving (i) a computer-based model of an object, (ii) a computer-based model of an environment, and (iii) an indication of position of a DHM in the environment;   determining an oriented bounding box surrounding the received model of the object, wherein the oriented bounding box includes a plurality of faces;   for each of the plurality of faces, determining: a candidate grasp location, a candidate grasp orientation, and a candidate grasp type;   from amongst the plurality of faces, determining one or more graspable faces based on: (a) the determined candidate grasp location of each face, (b) the determined candidate grasp orientation of each face, (c) the received model of the environment, and (d) dimensions of each face;   from amongst the determined one or more graspable faces, determining an optimal graspable face based on a predetermined grasping hierarchy and the received indication of position of the DHM in the environment; and   using an inverse kinematic solver to determine position and orientation of an end effector of the DHM grasping the object based on the determined candidate grasp location, the determined candidate grasp orientation, and the determined candidate grasp type of the determined optimal graspable face.   
     
     
         2 . The method of  claim 1  wherein determining the oriented bounding box comprises:
 determining a minimum bounding box surrounding the received model of the object; 
 determining a principal axis of inertia of the object based on the received model of the object; 
 orienting the determined minimum bounding box based on the determined principal axis of inertia; and 
 setting the oriented minimum bounding box as the oriented bounding box surrounding the received model of the object. 
 
     
     
         3 . The method of  claim 2  wherein determining a candidate grasp orientation for a given face of the plurality of faces comprises:
 setting the candidate grasp orientation for the given face based on the determined principal axis of inertia of the object. 
 
     
     
         4 . The method of  claim 1  wherein determining a candidate grasp location for a given face of the plurality of faces comprises:
 based on the received model of the object, calculating a geometrical center of the object; 
 projecting from the calculated geometrical center of the object to the given face; and 
 setting location of an intersection of the projection and the given face as the candidate grasp location for the given face. 
 
     
     
         5 . The method of  claim 1  wherein determining a candidate grasp type for a given face of the plurality of faces comprises:
 calculating length of a first edge and a second edge of the given face, wherein the first edge and the second edge are perpendicular to each other; 
 calculating length of a face edge normal to the first edge and the second edge; and 
 determining the candidate grasp type for the given face based on: (i) the calculated length of the first edge, (ii) the calculated length of the second edge, and (iii) the calculated length of the face edge normal to the first edge and the second edge. 
 
     
     
         6 . The method of  claim 1  wherein each determined candidate grasp type is one of: a pinch type, a medium-wrap type, and a precision sphere type. 
     
     
         7 . The method of  claim 1  wherein, determining one or more graspable faces based on: (a) the determined candidate grasp location of each face, (b) the determined candidate grasp orientation of each face, (c) the received model of the environment, and (d) the dimensions of each face comprises:
 identifying a given face as a graspable face if (i) the end effector of the DHM, at the determined candidate grasp location in the determined candidate grasp orientation, does not collide with an element in the model of the environment and (ii) dimensions of the given face do not exceed a threshold. 
 
     
     
         8 . The method of  claim 1  wherein the DHM includes a left end effector and a right end effector and the method further comprises:
 receiving an indication of the end effector, from amongst the left end effector and the right end effector, of the DHM grasping the object. 
 
     
     
         9 . The method of  claim 8  further comprising:
 selecting the predetermined grasping hierarchy based on the received indication of the end effector. 
 
     
     
         10 . The method of  claim 1  further comprising:
 configuring the inverse kinematic solver to have an unconstrained rotation degree of freedom along an axis normal to the determined optimal graspable face. 
 
     
     
         11 . The method of  claim 1  further comprising:
 applying a respective label to each face of the plurality of faces, wherein for each face the respective label is a function of position of the DHM in relation to the face. 
 
     
     
         12 . The method of  claim 11  wherein the predetermined grasping hierarchy indicates a preferred order of graspable faces as a function of each respective label. 
     
     
         13 . The method  claim 1  further comprising:
 simulating physical interaction between the DHM and the object using the determined position and orientation of the end effector. 
 
     
     
         14 . A system for determining position and orientation of an end effector of a digital human model (DHM) for grasping an object, the system comprising:
 a processor; and   a memory with computer code instructions stored thereon, the processor and the memory, with the computer code instructions, being configured to cause the system to:
 receive (i) a computer-based model of an object, (ii) a computer-based model of an environment, and (iii) an indication of position of a DHM in the environment; 
 determine an oriented bounding box surrounding the received model of the object, wherein the oriented bounding box includes a plurality of faces; 
 for each of the plurality of faces, determine: a candidate grasp location, a candidate grasp orientation, and a candidate grasp type; 
 from amongst the plurality of faces, determine one or more graspable faces based on: (a) the determined candidate grasp location of each face, (b) the determined candidate grasp orientation of each face, (c) the received model of the environment, and (d) dimensions of each face; 
 from amongst the determined one or more graspable faces, determine an optimal graspable face based on a predetermined grasping hierarchy and the received indication of position of the DHM in the environment; and 
 use an inverse kinematic solver to determine position and orientation of an end effector of the DHM grasping the object based on the determined candidate grasp location, the determined candidate grasp orientation, and the determined candidate grasp type of the determined optimal graspable face. 
   
     
     
         15 . The system of  claim 14  wherein:
 in determining the oriented bounding box, the processor and the memory, with the computer code instructions, are further configured to cause the system to:
 determine a minimum bounding box surrounding the received model of the object; 
 determine a principal axis of inertia of the object based on the received model of the object; 
 orient the determined minimum bounding box based on the determined principal axis of inertia; and 
 set the oriented minimum bounding box as the oriented bounding box surrounding the received model of the object; and 
 
 in determining a candidate grasp orientation for a given face of the plurality of faces, the processor and the memory, with the computer code instructions, are further configured to cause the system to:
 set the candidate grasp orientation for the given face based on the determined principal axis of inertia of the object. 
 
 
     
     
         16 . The system of  claim 14  wherein, in determining a candidate grasp location for a given face of the plurality of faces, the processor and the memory, with the computer code instructions, are further configured to cause the system to:
 based on the received model of the object, calculate a geometrical center of the object; 
 project from the calculated geometrical center of the object to the given face; and 
 set location of an intersection of the projection and the given face as the candidate grasp location for the given face. 
 
     
     
         17 . The system of  claim 14  wherein, in determining a candidate grasp type for a given face of the plurality of faces, the processor and the memory, with the computer code instructions, are further configured to cause the system to:
 calculate length of a first edge and a second edge of the given face, wherein the first edge and the second edge are perpendicular to each other; 
 calculate length of a face edge normal to the first edge and the second edge; and 
 determine the candidate grasp type for the given face based on: (i) the calculated length of the first edge, (ii) the calculated length of the second edge, and (iii) the calculated length of the face edge normal to the first edge and the second edge. 
 
     
     
         18 . The system of  claim 14  wherein, in determining one or more graspable faces based on: (a) the determined candidate grasp location of each face, (b) the determined candidate grasp orientation of each face, (c) the received model of the environment, and (d) the dimensions of each face, the processor and the memory, with computer code instructions, are further configured to cause the system to:
 identify a given face as a graspable face if (i) the end effector of the DHM, at the determined candidate grasp location in the determined candidate grasp orientation, does not collide with an element in the model of the environment and (ii) dimensions of the given face do not exceed a threshold. 
 
     
     
         19 . The system of  claim 14  wherein the processor and the memory, with the computer code instructions, are further configured to cause the system to:
 configure the inverse kinematic solver to have an unconstrained rotation degree of freedom along an axis normal to the determined optimal graspable face. 
 
     
     
         20 . A non-transitory computer program product for determining position and orientation of an end effector of a digital human model (DHM) for grasping an object, the computer program product executed by a server in communication across a network with one or more client and comprising:
 a computer readable medium, the computer readable medium comprising program instructions which, when executed by a processor, causes the processor to:
 receive (i) a computer-based model of an object, (ii) a computer-based model of an environment, and (iii) an indication of position of a DHM in the environment; 
 determine an oriented bounding box surrounding the received model of the object, wherein the oriented bounding box includes a plurality of faces; 
 for each of the plurality of faces, determine: a candidate grasp location, a candidate grasp orientation, and a candidate grasp type; 
 from amongst the plurality of faces, determine one or more graspable faces based on: (a) the determined candidate grasp location of each face, (b) the determined candidate grasp orientation of each face, (c) the received model of the environment, and (d) dimensions of each face; 
 from amongst the determined one or more graspable faces, determine an optimal graspable face based on a predetermined grasping hierarchy and the received indication of position of the DHM in the environment; and 
 use an inverse kinematic solver to determine position and orientation of an end effector of the DHM grasping the object based on the determined candidate grasp location, the determined candidate grasp orientation, and the determined candidate grasp type of the determined optimal graspable face.