Predictive robotic controller apparatus and methods

Abstract

Robotic devices may be trained by a user guiding the robot along target action trajectory using an input signal. A robotic device may comprise an adaptive controller configured to generate control signal based on one or more of the user guidance, sensory input, performance measure, and/or other information. Training may comprise a plurality of trials, wherein for a given context the user and the robot's controller may collaborate to develop an association between the context and the target action. Upon developing the association, the adaptive controller may be capable of generating the control signal and/or an action indication prior and/or in lieu of user input. The predictive control functionality attained by the controller may enable autonomous operation of robotic devices obviating a need for continuing user guidance.

1 . A computerized controller apparatus configured to effectuate control of a robotic device, the apparatus comprising:
 one or more processors configured to execute computer program modules to cause one or more processors to:
 (1) during a first plurality of training trials:
 determine a given first control signal of a plurality of first control signals based on (i) a characteristic of an input provided to the controller apparatus by the robotic device and (ii) a user input; and 
 cause the robotic device to perform a first action based on the given first control signal, the first action being characterized by a first performance; and 
 
 (2) during a training trial subsequent to the first plurality of training trials: 
 determine a second control signal based on the characteristic of the input and the first performance; and 
 cause the robotic device to perform a second action based on the second control signal; 
   wherein the second control signal is determined absent of a provision of user input subsequent to termination of the first plurality of training trials and prior to generation of the second control signal.   
     
     
         2 . The apparatus of  claim 1 , wherein:
 the first performance is determined based on a proximity measure between the first action and a target action; and   the second action is characterized by a second performance determined based on a proximity measure between the second action and the target action.   
     
     
         3 . A robotic apparatus, comprising:
 a platform comprising one or more controllable elements; and   a controller comprising one or more processors configured to execute computer program modules configured to operate individual ones of the one or more controllable elements, the computer program modules comprising:
 a first logic module configured to receive from a teacher a first teaching action indication of a plurality of teaching action indications; 
 a second logic module configured to cause the platform to perform an action in accordance with a first signal configured based on the first teaching action indication and a sensory signal; and 
 a third logic module configured to generate an action indication based on the sensory signal and analysis of individual ones of a plurality of first signals, the plurality of first signals being configured based on individual ones of the plurality of teaching action indications and the sensory signal; 
   wherein:
 the plurality of first signals comprises the first signal; and 
 the action indication is configured to cause the platform to perform the action in accordance with a second signal, the second signal being configured based on the action indication and the sensory signal. 
   
     
     
         4 . The apparatus of  claim 3 , wherein:
 the target action corresponds to operation of the platform based on teaching input from the teacher;   the analysis of individual ones of the plurality of first signals comprises determining of a deviation between the first action and a target action; and   the analysis is configured to cause modification of a controller state in accordance with a learning process, the learning process being configured based on a performance measure.   
     
     
         5 . The apparatus of  claim 3 , wherein:
 the second logic module comprises a predictor sub-module configured to determine a predicted control output based on a characteristic of the sensory signal, the sensory signal conveying information associated with one or both of an environment of the robotic apparatus and a platform state;   the first input and the second input are configured based on the sensory signal; and the action is configured based on the predicted control output.   
     
     
         6 . The apparatus of  claim 5 , wherein:
 the predictor sub-module is configured to provide a table configured to store a plurality of teaching inputs, a plurality of sensory signal characteristics, and a plurality of predicted control outputs; and   the analysis comprises selecting a given predicted control output based on a match between a given characteristic of sensory signal and an individual one of the plurality of sensory signal characteristics.   
     
     
         7 . The apparatus of  claim 6 , wherein the given predicted control is configured based on a search of the table, the search being configured based on the teaching input. 
     
     
         8 . The apparatus of  claim 5 , wherein:
 the third logic module comprises a combiner sub-module configured to determine a combined output based on the predicted control output and first user input, the combined output being characterized by a transform function configured to combine the predicted control output and the first signal via one or more operations including an additive operation.   
     
     
         9 . The apparatus of  claim 8 , wherein the transform function is configured to combine the predicted control output and the control output via one or more operations including a union operation. 
     
     
         10 . The apparatus of  claim 8 , wherein:
 the learning process comprises a supervised learning process configured based on the combined output;   the learning process is configured to be updated at time intervals; and   the modification of the controller is based on an error measure between (i) the predicted control output generated at a first time instance and (ii) the first input determined at second time instance subsequent to the first time instance, the first time instance and the second time instance separated by one of the time intervals.   
     
     
         11 . The apparatus of
   claim 3 , wherein: the teacher   comprises a human trainer;   the second input being configured to cause a corrective action by the platform, the action undergoing correction being characterized by a lower deviation from a target action;   the action undergoing correction is effectuated based on a cooperative interaction with the human, the cooperative interaction being characterized by a plurality of training iterations;   the first input corresponds to a first given iteration of the plurality of training iterations;   
       and
 the second input corresponds to a second given iteration of the plurality of training iterations, the second given iteration occurring subsequent to the first given iteration. 
 
     
     
         12 . The apparatus of  claim 11 , wherein:
 the second logic module comprises a predictor sub-module configured to determine a plurality of predicted control outputs based on a characteristic of a sensory signal and a given user input;   the sensory signal is configured to convey information about one or both of an environment of the robotic apparatus and a plant state;   the first input and the second input are configured based on the sensory signal;   the action is configured based on a first predicted control output of the plurality of predicted control outputs, the first predicted control output corresponding to the first input;   the corrected action is configured based on a second predicted control output of the plurality of predicted control outputs, the second predicted control output corresponding to the second input;   the corrected action being characterized by an improved performance as compared to the action and a target action;   the improved performance being quantified based on a lower deviation of the corrected action from the target action compared to a deviation between the action and the target action; and   the target action being based solely on a training input by the human absent a predicted control output.   
     
     
         13 . The apparatus of  claim 3 , wherein the second signal is generated by the controller based on a plurality of action indications being generated by the third logic and absent an explicit indication by the teacher. 
     
     
         14 . The apparatus of  claim 3 , wherein the second signal is generated by the controller responsive to a confirmation provided by the teacher, the second signal being generated based on a confirmation indication from the teacher. 
     
     
         15 . A method of training a computerized robotic apparatus to perform a target action based on sensory context, the training being effectuated via a plurality of iterations, the method 
       comprising:
 during a first portion of the plurality of iterations:
 causing the apparatus to perform an action based on a first control signal generated based on the sensory context and a target action indication provided by a user; and 
 causing the apparatus to adjust a controller learning parameter based on a first performance determined based on the first action and the target action; and 
 
 during a second portion of the plurality of iterations, the second portion being subsequent to the first portion:
 causing the apparatus to generate an action signal based on the sensory context and the adjusted learning parameter, the action signal generation occurring prior to provision of the target action signal associated with the subsequent iteration; and 
 causing the apparatus to perform the action based on a second control signal generated based on the sensory context and the action signal but absent the provision of the target action indication by the user; and 
 
 wherein:
 the action performance based on the second control signal is characterized by a second performance; 
 provision of the target action indication by the user during the second portion of the plurality of iterations is configured to cause performance execution of another action characterized by a third performance; and 
 the third performance value is lower than the second performance value. 
 
 
     
     
         16 . The method of  claim 15 , wherein:
 the performance measure being determined by the controller; and   the third performance value is lower than the first performance value.   
     
     
         17 . The method of  claim 15 , wherein:
 the controller learning parameter adjustment is configured based on a supervised learning process configured based on the sensory context and a combination of the control signal and the user input; and   the third performance value is lower than the second performance value.   
     
     
         18 . The method of  claim 15 , wherein:
 the sensory context comprises an object representation; and   the target action comprises at least one of an object approach maneuver or an object avoidance maneuver.   
     
     
         19 . The method of  claim 15 , wherein:
 individual ones of the first portion of the plurality of iterations are characterized by a time interval between an onset of the sensory context and provision of the user input; and   a time period between the onset of the sensory context during the second portion of the plurality of iterations and the action signal generation is no greater than a minimum value of the time interval.   
     
     
         20 . The method of  claim 15 , wherein:
 individual ones of the first portion of the plurality of iterations are characterized by a time interval between an onset of the sensory context and provision of the user input; and   a time period between the onset of the sensory context during the second portion of the plurality of iterations iteration and the action signal generation is no less than at least one of a mean value of the time interval or a median value of the time interval.   
     
     
         21 . (canceled) 
     
     
         22 . (canceled)