Facilitating human intervention in an autonomous device

Abstract

Methods, apparatuses, systems, and computer program products for facilitating human intervention in an autonomous device are disclosed. In a particular embodiment, a method of facilitating human intervention in an autonomous device includes a service controller selecting from a first plurality of human interventionists, by a service controller, a first set of human interventionists to respond to a request associated with an autonomous device; transmitting, by the service controller, the request to a first set of interventionist devices, each interventionist device of the first set of interventionist devices associated with a particular human interventionist in the first set of human interventionists; and receiving from the first set of interventionist devices, by the service controller, a first set of interventionist responses to the request.

What is claimed is: 
     
         1 . A method for facilitating human intervention in an autonomous device, the method comprising:
 selecting from a first plurality of human interventionists, by a service controller, a first set of human interventionists to respond to a request associated with an autonomous device;   transmitting, by the service controller, the request to a first set of interventionist devices, each interventionist device of the first set of interventionist devices associated with a particular human interventionist in the first set of human interventionists; and   receiving from the first set of interventionist devices, by the service controller, a first set of interventionist responses to the request.   
     
     
         2 . The method of  claim 1  further comprising:
 based on the first set of interventionist responses, generating, by the service controller, a service response to the request; and 
 transmitting, by the service controller, the service response to the autonomous device. 
 
     
     
         3 . The method of  claim 1  further comprising:
 determining, by the service controller, whether the service controller is sufficiently confident in the first set of interventionist responses; 
 in response to determining that the service controller is sufficiently confident in the first set of interventionist responses:
 based on the first set of interventionist responses, generating, by the service controller, a service response to the request; and 
 transmitting, by the service controller, the service response to the autonomous device; and 
 
 in response to determining that the service controller is not sufficiently confident in the first set of interventionist responses:
 selecting from a second plurality of human interventionists, by the service controller, a second set of human interventionists to respond to the request; 
 transmitting, by the service controller, the request to a second set of interventionist devices, each interventionist device of the second set of interventionist devices associated with a particular human interventionist in the second set of human interventionists; and 
 receiving, by the service controller, a second set of interventionist responses from the second set of interventionist devices. 
 
 
     
     
         4 . The method of  claim 3  further comprising:
 determining, by the service controller, whether the service controller is sufficiently confident in an aggregate of the first set of interventionist responses and the second set of interventionist responses; 
 in response to determining that the service controller is sufficiently confident in the aggregate of the first set of interventionist responses and the second set of interventionist responses:
 based on the aggregate of the first set of interventionist responses and the second set of interventionist responses, generating, by the service controller, a service response to the request; and 
 transmitting, by the service controller, the service response to the autonomous device; and 
 
 in response to determining that the service controller is not sufficiently confident in the aggregate of the first set of interventionist responses and the second set of interventionist responses:
 selecting from a third plurality of human interventionists, by the service controller, a third set of human interventionists to respond to the request; 
 transmitting, by the service controller, the request to a third set of interventionist devices, each interventionist device of the third set of interventionist devices associated with a particular human interventionist in the third set of human interventionists; and 
 receiving, by the service controller, a third set of interventionist responses from the third set of interventionist devices. 
 
 
     
     
         5 . The method of  claim 1  further comprises:
 determining, by the service controller, whether the service controller can perform the request autonomously; and 
 wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request is in response to determining that the service controller cannot perform the request autonomously; 
 wherein transmitting, by the service controller, the request to the first set of interventionist devices is in response to determining that the service controller cannot perform the request autonomously; and 
 wherein receiving from the first set of interventionist devices, by the service controller, the first set of interventionist responses to the request is in response to determining that the service controller cannot perform the request autonomously. 
 
     
     
         6 . The method of  claim 5  further comprising:
 in response to determining that the service controller can perform the request autonomously, generating, by the service controller, an autonomous response. 
 
     
     
         7 . The method of  claim 6  further comprising:
 based on the autonomous response, generating, by the service controller, a service response to the request; and 
 transmitting, by the service controller, the service response to the autonomous device. 
 
     
     
         8 . The method of  claim 1  further comprising:
 determining, by the service controller, whether the service controller can perform the request autonomously; 
 in response to determining that the service controller can perform the request autonomously, generating, by the service controller, an autonomous response. 
 determining, by the service controller, whether the service controller is sufficiently confident in the autonomous response; 
 in response to determining that the service controller is sufficiently confident in the autonomous response:
 based on the autonomous response, generating, by the service controller, a service response to the request; and 
 transmitting, by the service controller, the service response to the autonomous device; and 
 
 wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request is in response to determining that that the service controller is not sufficiently confident in the autonomous response; 
 wherein transmitting, by the service controller, the request to the first set of interventionist devices is in response to determining that that the service controller is not sufficiently confident in the autonomous response; and 
 wherein receiving from the first set of interventionist devices, by the service controller, the first set of interventionist responses to the request is in response to determining that that the service controller is not sufficiently confident in the autonomous response. 
 
     
     
         9 . The method of  claim 8  further comprising:
 determining, by the service controller, whether the service controller is sufficiently confident in an aggregate of the autonomous response and the first set of interventionist responses; 
 in response determining that the service controller is sufficiently confident in the aggregate of the autonomous response and the first set of interventionist responses:
 based on one of the first set of interventionist response and the aggregate of the autonomous response and the first set of interventionist responses, generating, by the service controller, a service response to the request; and 
 transmitting, by the service controller, the service response to the autonomous device; and 
 
 in response to determining that the service controller is not sufficiently confident in the aggregate of the autonomous response and the first set of interventionist responses:
 selecting from a second plurality of human interventionists, by the service controller, a second set of human interventionists to respond to the request; 
 transmitting, by the service controller, the request to a second set of interventionist devices, each interventionist device of the second set of interventionist devices associated with a particular human interventionist in the second set of human interventionists; and 
 receiving, by the service controller, a second set of interventionist responses from the second set of interventionist devices. 
 
 
     
     
         10 . The method of  claim 1 , wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 determining, for the request, an expense range and selection criteria;   for one or more human interventionists of the first plurality of human interventionists, determining an interventionist expense for each of the one or more human interventionists, the interventionist expense indicating the cost of using the human interventionist to respond to the request; and   selecting, from the first plurality of human interventionists, the first set of human interventionists based on the expense range, the selection criteria, and the interventionist expense of each of the one or more human interventionists.   
     
     
         11 . The method of  claim 1 , wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 determining, for the request, a difficulty rating and selection criteria;   for one or more human interventionists of the first plurality of human interventionists, determining a qualification rating of each of the one or more human interventionists; and   selecting, from the first plurality of human interventionists, the first set of human interventionists based on the difficulty rating, the selection criteria, and the qualification rating of each of the one or more human interventionists.   
     
     
         12 . The method of  claim 1  wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 for one or more human interventionists of the first plurality of human interventionists, determining a responsiveness of each of the one or more human interventionists; 
 determining, for the request, an urgency rating and selection criteria; and 
 selecting, from the first plurality of human interventionists, the first set of human interventionists based on the urgency rating, the selection criteria, and the responsiveness of each of the one or more human interventionists. 
 
     
     
         13 . The method of  claim 1  wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 for one or more human interventionists of the first plurality of human interventionists, determining, by the service controller, a performance metric of each of the one or more human interventionists; 
 determining, for the request, a performance requirement and selection criteria; and 
 selecting, from the first plurality of human interventionists, the first set of human interventionists based on the performance requirement, the selection criteria, and the performance metric of each of the one or more human interventionists. 
 
     
     
         14 . The method of  claim 1  wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 determining, by the service controller, a confidence requirement associated with the request; 
 determining a number of human interventionists needed to satisfy the confidence requirement; and 
 including at least the determined number of human interventionists within the first set of human interventionists. 
 
     
     
         15 . The method of  claim 1  wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 determining a type of task to be performed, by a human interventionist, for responding to the request; 
 determining a type of interventionist device for performing the task; and 
 selecting for the first set of human interventionists, a particular human interventionist having a particular interventionist device matching the determined type of interventionist device. 
 
     
     
         16 . The method of  claim 1  wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 determining a type of task to be performed by a human interventionist, for responding the request; and 
 selecting for the first set of human interventionists, a particular human interventionist having a profile indicating the capability of performing the type of task to be performed for responding to the request. 
 
     
     
         17 . The method of  claim 1  wherein each human interventionist in the first set of human interventionists has an associated expense for performing a particular type of request; the method further comprising:
 attributing to an account associated with the autonomous device, by the service controller, the particular expense associated with each human interventionist in the first set of human interventionists. 
 
     
     
         18 . The method of  claim 1  further comprising:
 for one or more human interventionists in the first plurality of human interventionists:
 transmitting, by the service controller, a test request to a particular interventionist device associated with a particular human interventionist; 
 receiving from the particular interventionist device, by the service controller, a test response to the transmitted test request; and 
 based on the test response, generating, by the service controller, a performance metric of the human interventionist; and 
 
 based on the performance metric of each of the one or more human interventionists of the first plurality of human interventionists, determining, by the service controller, a rating for each of the one or more human interventionists; and 
 wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes selecting, based on the determined ratings, the first set of human interventionists. 
 
     
     
         19 . The method of  claim 18 , wherein based on the test response, generating, by the service controller, the performance metric of the human interventionist includes:
 transmitting the test response to a human interventionist evaluator; and   receiving from the human interventionist evaluator, an evaluation of the test response.   
     
     
         20 . The method of  claim 1  wherein selecting from the first plurality of human interventionists, by the service controller, the first set of human interventionists to respond to the request includes:
 determining, by the service controller, whether a particular human interventionist is available; 
 in response to determining that the particular human interventionist is not available, excluding, by the service controller, the particular human interventionist from the first set of human interventionists to respond to the request; and 
 in response to determining that the particular human interventionist is available, adding, by the service controller, the particular human interventionist to the first set of human interventionists to respond to the request. 
 
     
     
         21 . A computer readable medium for facilitating human intervention in an autonomous device, the computer readable medium including a computer program instructions that when executed by a computer processor cause the computer processor to perform:
 for each human interventionist of a plurality of human interventionists:
 transmitting, by a service controller, a test request to a particular interventionist device associated with a particular human interventionist; 
 receiving from the particular interventionist device, by the service controller, a test response to the transmitted test request; and 
 based on the test response, generating, by the service controller, a performance metric of the human interventionist; and 
   based on the performance metrics of the human interventionists of the plurality of human interventionists, determining, by the service controller, a rating for each human interventionist of the plurality of human interventionists.   
     
     
         22 . The computer readable medium of  claim 21  further comprising computer program instructions that when executed by the computer processor cause the computer processor to perform:
 receiving, by the service controller, a request from an autonomous device; 
 selecting from the plurality of human interventionists, by the service controller, a first set of human interventionists to respond to the request; 
 transmitting, by the service controller, the request to a first set of interventionist devices, each interventionist device of the first set of interventionist devices associated with a particular human interventionist in the first set of human interventionists; and 
 receiving from the first set of interventionist devices, by the service controller, a first set of interventionist responses to the request. 
 
     
     
         23 . A method of facilitating human intervention in an autonomous device, the method comprising:
 placing, by an autonomous device, an object in a stack of one or more objects;   in response to placing the object in the stack, detecting, by the autonomous device, instability in the stack;   determining, by the autonomous device, that correcting the instability requires a level of capability beyond a threshold associated with the autonomous device; and   in response to determining that correcting the instability requires the level of capability beyond the threshold associated with the autonomous device, generating, by the autonomous device, a request for human intervention.   
     
     
         24 . A method of facilitating human intervention in an autonomous device, the method comprising:
 grasping, by an autonomous device, an object with one or more components of the autonomous device;   based on sensor input, determining, by the autonomous device, a level of confidence that the object is securely grasped with the one or more components of the autonomous device;   determining, by the autonomous device, that the level does not satisfy a confidence requirement; and   in response to determining that the level does not satisfy the confidence requirement, generating, by the autonomous device, a request for human intervention.   
     
     
         25 . A method of facilitating human intervention in an autonomous device, the method comprising:
 receiving, by a service controller, a request from an autonomous device;   generating, by the service controller, a service response to the request, the service response generated based on least one of:
 a first set of interventionist responses received from a first set of interventionist devices associated with a first set of human interventionists; and 
 a first set of autonomous responses generated by an autonomous module; and 
   training, by the service controller, the autonomous module using at least one of the first set of interventionist responses and the first set of autonomous responses.   
     
     
         26 . The method of  claim 25  further comprising transmitting, by the service controller, the service response to the autonomous device.