Adapting object handover from robot to human using perceptual affordances

Abstract

This disclosure describes, according to some implementations, a system and method for adapting object handover from robot to human using perceptual affordances. In an example method, upon receiving sensor data describing surroundings and/or operational state of a robot unit from a sensor, the method calculates a probability of a perceptual classification based on the sensor data. The perceptual classification may be one or more of an environment classification, an object classification, a human action classification, or an electro-mechanical state classification. The method further calculates an affordance of the probability of the perceptual classifier using a preference model, determines a handover action based on the affordance, executes the handover action, and updates the preference model based on feedback.

What is claimed is: 
     
       1. A computer-implemented method for executing a handover action, the method comprising:
 receiving, using one or more computer processors, sensor data from one or more sensors coupled to a robot unit, the sensor data describing one or more of surroundings of the robot unit and an object manipulated by the robot unit; 
 calculating, using the one or more computer processors, a probability of a perceptual classification related to one or more of the robot unit, a human action, an environment, an electro-mechanical state based on the sensor data; 
 calculating, using the one or more computer processors, an affordance based on the probability of the perceptual classification using a preference model; 
 determining, using the one or more computer processors, a handover action based on the affordance; 
 executing, using the one or more computer processors, the handover action; and 
 updating, using the one or more computer processors, the preference model based on feedback. 
 
     
     
       2. A computer-implemented method for executing a handover action, the method comprising:
 receiving, using one or more computer processors, sensor data from one or more sensors; 
 calculating, using the one or more computer processors, a probability of a perceptual classification based on the sensor data; 
 calculating, using the one or more computer processors, an affordance based on the probability of the perceptual classification using a preference model; 
 determining, using the one or more computer processors, a handover action based on the affordance; 
 executing, using the one or more computer processors, the handover action; and 
 updating, using the one or more computer processors, the preference model based on feedback. 
 
     
     
       3. The method of  claim 2 , wherein calculating the probability of the perceptual classification further comprises:
 extracting, using the one or more processors, object data from the sensor data; 
 determining, using the one or more processors, an object classification based on the object data; and 
 calculating, using the one or more processors, the probability of the perceptual classification using the object classification. 
 
     
     
       4. The method of  claim 2 , wherein calculating the probability of the perceptual classification further comprises:
 extracting, using the one or more processors, environment data from the sensor data; 
 determining, using the one or more processors, an environment classification based on the environment data; and 
 calculating, using the one or more processors, the probability of the perceptual classification using the environment classification. 
 
     
     
       5. The method of  claim 4 , wherein the environment data is a geolocation. 
     
     
       6. The method of  claim 2 , wherein calculating the probability of the perceptual classification further comprises:
 extracting, using the one or more processors, activity data from the sensor data; 
 determining, using the one or more processors, a human activity classification based on the activity data; and 
 calculating, using the one or more processors, the probability of the perceptual classification using the human activity classification. 
 
     
     
       7. The method of  claim 2 , wherein calculating the probability of the perceptual classification further comprises:
 extracting, using the one or more processors, state data from the sensor data; 
 determining, using the one or more processors, an electro-mechanical state classification based on the state data; and 
 calculating, using the one or more processors, the probability of the perceptual classification using the electro-mechanical state classification. 
 
     
     
       8. The method of  claim 2 , wherein updating the preference model based on the feedback further comprises:
 monitoring, using the one or more processors, the executed handover action; 
 determining, using the one or more processors, a success of the handover action; 
 determining, using the one or more processors, an indirect feedback based on the success; and 
 updating, using the one or more processors, the preference model using the indirect feedback. 
 
     
     
       9. The method of  claim 2 , wherein updating the preference model based on the feedback further comprises:
 requesting, using the one or more processors, a direct feedback from a user; and 
 updating, using the one or more processors, the preference model using the direct feedback. 
 
     
     
       10. The method of  claim 2 , wherein calculating the probability of the perceptual classification based on the sensor data includes calculating a first probability for an object classification, calculating a second probability for a human activity, calculating a third probability for an environment, and calculating a fourth probability for an electro-mechanical state classification, and calculating the affordance based on the first probability, the second probability, the third probability, and the fourth probability. 
     
     
       11. A system comprising:
 one or more computer processors; and 
 one or more memories storing instructions that, when executed by the one or more processors, cause the system to perform operations including:
 receive sensor data from a sensor, the sensor data describing surroundings; 
 calculate a probability of a perceptual classification based on the sensor data; 
 calculate an affordance of the probability of the perceptual classification based on a preference model; 
 determine a handover action based on the affordance; 
 execute the handover action; and 
 updating the preference model based on feedback. 
 
 
     
     
       12. The system of  claim 11 , wherein the instructions to calculate the probability of the perceptual classification further include:
 extract object data from the sensor data; 
 determine an object classification based on the object data; and 
 calculate the probability of the perceptual classification using the object classification. 
 
     
     
       13. The system of  claim 11 , wherein the instructions to calculate the probability of the perceptual classification further include:
 extract environment data from the sensor data; 
 determine an environment classification based on the environment data; and 
 calculate the probability of the perceptual classification using the environment classification. 
 
     
     
       14. The system of  claim 13 , wherein the environment data is a geolocation. 
     
     
       15. The system of  claim 11 , wherein the instructions to calculate the probability of the perceptual classification further include:
 extract activity data from the sensor data; 
 determine a human activity classification based on the activity data; and 
 calculate the probability of the perceptual classification using the human activity classification. 
 
     
     
       16. The system of  claim 11 , wherein the instructions to calculate the probability of the perceptual classification further include:
 extract state data from the sensor data; 
 determine an electro-mechanical state classification based on the state data; and 
 calculate the probability of the perceptual classification using the electro-mechanical state classification. 
 
     
     
       17. The system of  claim 11 , wherein the instruction to update the preference model based on the feedback further include:
 monitor the executed handover action; 
 determine a success of the handover action; 
 determine an indirect feedback based on the success; and 
 update the preference model using the indirect feedback. 
 
     
     
       18. The system of  claim 11 , wherein the instructions to update the preference model based on the feedback further include:
 request a direct feedback from a user; and 
 update the preference model using the direct feedback. 
 
     
     
       19. The system of  claim 11 , wherein calculating the probability of the perceptual classification based on the sensor data includes calculating a first probability for an object classification, calculating a second probability for a human activity, calculating a third probability for an environment, and calculating a fourth probability for an electro-mechanical state classification, and calculating the affordance based on the first probability, the second probability, the third probability, and the fourth probability.