Visual Interface And Communications Techniques For Use With Robots

Abstract

A humanoid robot may include one or two eye resembling features in the form of image sensor(s) that capture images of at least a portion of an environment. The fields-of-view of the image sensor(s) will change as a pose (i.e., position, orientation, position and orientation) of the head, torso, or even entire humanoid robot changes. Virtual or augmented representations of the environment may be presented to a human pilot of the robot, where disorienting feeling due to latency in communications can be alleviated by reducing demand on communications channels.

1 . A method of operation in a processor-based system to predict physical movement in an environment in which a robot operates, the environment including one or more objects, the method comprising:
 generating a first digital representation of at least a portion of the environment in which the robot operates, the first digital representation including, for each of the one or more objects, a respective three dimensional spatial representation of the object and at least one non-spatial physical characteristic of the object;   performing a virtual physics simulation on the first digital representation of the at least portion of the environment in which the robot operates; and   for at least one of the one or more objects, predicting at least one of a movement of the respective object or a collision of the respective object with another one of the objects based on the virtual physics simulation.   
     
     
         2 . The method of  claim 1  wherein performing a virtual physics simulation includes iteratively evaluating at least a position and a velocity of each of the one or more objects based at least in part on the three dimensional spatial representation of the one or more objects and at least one non-spatial physical characteristic of the one or more objects. 
     
     
         3 . The method of  claim 2  wherein iteratively evaluating at least a position and a velocity of each of the one or more objects includes comparing at least one of: one or more positions, one or more poses, one or more velocities, one or more accelerations, one or more forces, one or more masses, one or more volumes, one or more orientations, one or more momentums, or one or more torques at each iteration in the virtual physics simulation with at least one of: one or more positions, one or more poses, one or more velocities, one or more accelerations, one or more forces, one or more masses, one or more volumes, one or more orientations, one or more momentums, or one or more torques at a corresponding point of time. 
     
     
         4 . The method of  claim 1 , further comprising:
 generating at least a second digital representation of at least the portion of the environment in which the robot operates, the second digital representation including, for each of the one or more objects, a respective three dimensional spatial representation of the object and at least one non-spatial physical characteristic of object, where at least one of the respective three dimensional spatial representation of the object or the at least one non-spatial physical characteristic of object represented in the second digital representation is different from the corresponding respective three dimensional spatial representation of the object or the at least one non-spatial physical characteristic of object represented in the first digital representation;   performing a virtual physics simulation on at least the second digital representation of the at least portion of the environment in which the robot operates; and   for at least one of the one or more objects, predicting at least one of a movement of the respective object or a collision of the respective object with another one of the objects based on at least one of the virtual physics simulation subsequent to the virtual physics simulation performed on the first digital representation.   
     
     
         5 . The method of  claim 1 , further comprising:
 generating a virtual representation of the robot including any associated limbs, joints, tools, or appendages that are a part of the robot; and   simulating at least one movement of at least a portion of the robot within the digital representation of the at least the portion of the environment in which the robot operates.   
     
     
         6 . A method of operation in a processor-based system to predict and drive a robot, the method comprising:
 simulating several instances of a digital representation of the robot with respective sets of movements of at least some of a number of joints or a number of actuators of the robot, the sets of movements differing from one another; and   selecting one of the sets of movements that results in achievement of a goal state of the robot.   
     
     
         7 . The method of  claim 6  wherein selecting one of the sets of movements that results in achievement of a goal state of the robot includes: selecting two or more sets of movements that result in achievement of the goal state of the robot, and selecting from the two or more sets of movements that result in achievement of the goal state of the robot a one of the sets of movement that also incurs a least amount of work by the robot to achieve the goal state. 
     
     
         8 . The method of  claim 6  wherein selecting one of the sets of movements that results in achievement of a goal state of the robot includes: selecting two or more sets of movements that result in achievement of the goal state of the robot, and selecting from the two or more sets of movements that result in achievement of the goal state of the robot a one of the sets of movement that also takes the least elapsed time to reach the goal state. 
     
     
         9 . The method of  claim 6  wherein selecting one of the sets of movements that results in achievement of a goal state of the robot includes: selecting two or more sets of movements that result in achievement of the goal state of the robot, and providing the two or more sets of movement to a human operator for selection. 
     
     
         10 . The method of  claim 6 , further comprising:
 repeating the simulating and selecting at a plurality of different points throughout the movement to the goal state in order to account for random deviations and inaccuracies in the movement.   
     
     
         11 . A method of operation in a processor-based system to implement a visual interface with a robot, the method comprising:
 generating a sequence of digital representations of at least a portion of an environment in which the robot operates from at least one set of image information that represents at least one view of at least the portion of the environment in which the robot operates at sequential times, the at least one view corresponding to a field-of-view encompassed by at least one image sensor that is coupled to or a part of the robot; and   for at least one subsequent digital representation in the sequence of digital representations, generating at least one update digital representation, the update digital representation which represents a subset of information from at least one previous digital representation in the sequence, the subset of information representative of changes that occurred in the environment between the respective previous and the respective subsequent digital representations.   
     
     
         12 . The method of  claim 11 , further comprising:
 transmitting at least one previous digital representation to a user interface device for presentation of a virtual representation of the environment at a first time; and   subsequently, transmitting at least one update digital representation to the user interface device for presentation an updated virtual representation of the environment at a second time.   
     
     
         13 . The method of  claim 12  wherein generating at least one update digital representation includes generating a plurality of update digital representations, where a first one of the update digital representations represent changes with respect to a baseline digital representation, and each subsequent ones of the update digital representations represent changes with respect to a respective most immediately preceding update digital representation. 
     
     
         14 . The method of  claim 12  wherein generating at least one update digital representation includes generating a plurality of update digital representations, where a first one of the update digital representations represent changes with respect to a baseline digital representation, and each subsequent ones of the update digital representations represent changes with respect to the baseline digital representation. 
     
     
         15 . The method of  claim 12  wherein generating at least one update digital representation comprises: for each update digital representation, determining a number of changes in the environment between a subsequent time and a previous time, and only representing the determined changes in the respective update digital representation. 
     
     
         16 . The method of  claim 12  wherein generating a sequence of digital representations of at least a portion of an environment in which the robot operates from at least one set of image information comprises: generating a three-dimensional mapping of one or more portions of the environment including at least one of: one or more objects, one or more surfaces, one or more obstacles, one or more humans, or one or more animals present in the environment. 
     
     
         17 . The method of  claim 11 , further comprising:
 causing at least a first visual presentation of a first virtual representation of the environment in which the robot operates for a first period of time, the first virtual representation based at least in part on a first one of the digital representations in the sequence of digital representations of the environment, and the first period of time being longer than a period of time represented by the first one of the digital representations in the sequence of digital representations of the environment; and   causing at least a second visual presentation of a second virtual representation of the environment in which the robot operates for a second period of time, the second virtual representation based at least in part on a second one of the digital representations in the sequence of digital representations of the environment, and the second period of time being longer than a period of time represented by the second one of the digital representations in the sequence of digital representations of the environment.   
     
     
         18 . The method of  claim 17  wherein causing at least a second visual presentation of a second virtual representation of the environment in which the robot operates for a second period of time includes causing at least the second visual presentation of the second virtual representation of the environment based at least in part on a respective one of the update digital representations that represents changes that occurred in the environment between the respective first and the respective second digital representations of the environment.