Robot for participating in a joint performance with a human partner

Abstract

For a joint performance of a dialogue between a human partner and a robot, the robot analyzes phrase and action of the partner to detect a recognized behavior of the partner and analyzes a state of audience listening to utterances from the partner and the robot to detect a recognized state of the audience. A scenario describing the dialogue is stored in entries of a memory. The memory is successively referenced entry by entry and a check is made for a match between an utterance by the partner or the robot to a reaction from the audience. Responsive to a currently detected audience state, a corresponding robot behavior is determined. Preferably, possible partner's behaviors and expected audience states are mapped in a database to specified robot behaviors. The database is searched for a specified robot behavior corresponding to a currently sensed partner behavior or a currently sensed audience state.

1 - 13 . (canceled) 
   
   
       14 . A robot control system for controlling a robot, comprising:
 a first analyzer for analyzing phrase and action of a human partner to detect a recognized behavior of the partner;   a second analyzer for analyzing a state of a plurality of people listening to utterances from said partner and said robot to detect a recognized state of said people;   a scenario memory for storing a scenario describing a dialogue between said partner and said robot; and   a processor for making reference to a portion of said scenario in said memory according to the recognized behavior of said partner and the recognized state of said people, and determining a behavior of said robot according to the referenced portion of said scenario.   
   
   
       15 . The robot control system of  claim 14 , wherein said scenario memory includes a plurality of entries, each of said entries containing a portion of said scenario to be uttered by one of said partner and said robot,
 wherein said processor successively makes reference to each of said entries, produces a predetermined utterance from the robot according to the referenced portion of said scenario and determines said behavior of said robot according to one of a recognized behavior of said partner currently detected by the first analyzer and a recognized state of said people currently detected by the second analyzer.   
   
   
       16 . The robot control system of  claim 15 , further comprising a robot behavior database for mapping a plurality of recognized behaviors of said partner which will be detected by said first analyzer to a plurality of predetermined robot behaviors,
 wherein the referenced entry indicates an expected state of said people, and   wherein said processor compares a recognized state of said people currently detected by said second analyzer to the expected state of the referenced entry for a match or mismatch and determines said behavior of the robot according to the referenced portion of said scenario if there is a match between said recognized and expected states and according to one of said predetermined robot behaviors of said database corresponding to a recognized behavior of said partner currently detected by said first analyzer if there is a mismatch between said recognized and expected states.   
   
   
       17 . The robot control of  claim 15 , wherein each of said entries of said scenario memory indicates which one of said partner and said robot has the right to speak, and
 wherein said processor determines said behavior of said robot according to a recognized behavior of said partner currently detected by said first analyzer if the referenced entry indicates that said partner has the right to speak and according to a state of said people currently recognized by said second analyzer if said referenced entry indicates that said robot has the right to speak.   
   
   
       18 . The robot control system of  claim 17 , wherein said processor compares a recognized behavior of said partner currently detected by the first analyzer to a predetermined behavior if the referenced entry indicates that said partner has the right to speak, and produces an utterance from the robot for urging the partner to perform said predetermined behavior if there is a mismatch between said recognized behavior said predetermined behavior. 
   
   
       19 . The robot control system of  claim 16 , wherein each of said entries of said scenario memory indicates which one of said partner and said robot has the right to speak,
 wherein said database further maps a plurality of recognized states of said people which will be detected by said second analyzer to said plurality of predetermined robot behaviors,   wherein said processor determines said behavior of said robot according to one of said predetermined robot behaviors corresponding to a recognized behavior of said partner currently detected by said first analyzer if the referenced entry indicates that said partner has the right to speak and according to one of said predetermined robot behaviors corresponding to a recognized state of said people currently detected by said second analyzer if the referenced entry indicates that said robot has the right to speak.   
   
   
       20 . The robot control system of  claim 19 , wherein said processor compares a recognized behavior of said partner currently detected by the first analyzer to a predetermined partner's behavior if the referenced entry indicates that said partner has the right to speak, and produces an utterance from the robot for urging the partner to perform said predetermined partner's behavior if there is a mismatch between said recognized behavior said predetermined partner's behavior. 
   
   
       21 . The robot control system of  claim 15 , further comprising a robot behavior database for mapping a plurality of recognized states of said people which will be detected by said second analyzer to a plurality of predetermined robot behaviors, and
 wherein said processor determines said behavior of the robot according to one of said predetermined robot behaviors of said database corresponding to a recognized state of said people currently detected by said second analyzer.   
   
   
       22 . The robot control system of  claim 14 , wherein said dialogue stored in said scenario memory is a comic dialogue. 
   
   
       23 . The robot control system of  claim 14 , wherein said first analyzer recognizes the behavior of said partner from a microphone and a video camera. 
   
   
       24 . The robot control system of  claim 14 , wherein said second analyzer recognizes the state of said people from a microphone and a video camera. 
   
   
       25 . The robot control system of  claim 14 , wherein said first analyzer recognizes the state of said partner from a thermographic sensor and a touch sensor for producing a signal when the touch sensor is in contact with said partner. 
   
   
       26 . The robot control system of  claim 24 , wherein said robot has a torso and a head rotatable on said torso about a vertical axis, and wherein said video camera is mounted on said head. 
   
   
       27 . (canceled) 
   
   
       28 . The robot control system of  claim 14 , further comprising:
 a video screen located external to said robot;   a plurality of computer terminals located external to said robot;   a network for establishing a wireless link from said processor to said video screen and a wireless link from said computer terminals to said processor; and   a teaching manual memory for storing teaching materials,   wherein said processor reads a teaching material from said teaching manual memory and transmitting the material to said video screen and collecting data from said computer terminals.   
   
   
       29 - 46 . (canceled)