Enhanced video conference management

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enhanced video conference management. In some implementations, a media stream is received from each of multiple endpoint devices over a communication network. A video conference session among the endpoint devices is managed such that at least one or more of the media streams are transmitted over the communication network for display by the endpoint devices. A plurality of audio and/or video characteristics from the media stream from a particular endpoint device of the multiple endpoint devices are measured. Based on the audio and/or video characteristics, a collaboration factor score is determined for the particular endpoint device for each of a plurality of collaboration factors. The video conference of the endpoint devices by performing a video conference management action selected based on the collaboration factor scores.

What is claimed is: 
     
         1 . A method performed by one or more computers, the method comprising:
 receiving, by one or more computers, one or more media streams of a communication session involving multiple endpoint devices, the one or more media streams being received during the communication session;   determining, by the one or more computers, a speaking time duration for each of one or more participants in the communication session based on the one or more media streams of the communication session;   generating, by the one or more computers, data comprising one or more indicators of levels of participation of the one or more participants in the communication session based on the determined speaking time duration; and   providing, by the one or more computers, the generated data comprising the one or more indicators for presentation during the communication session.   
     
     
         2 . The method of  claim 1 , selecting a management action for the communication session from among a plurality of different management actions, wherein the selected management action is configured to alter transmission or presentation of the one or more media streams for one or more of the endpoint devices; and
 carrying out the selected management action or providing a recommendation for the selected management action.   
     
     
         3 . The method of  claim 1 , wherein the communication session is an audio conference. 
     
     
         4 . The method of  claim 1 , wherein the communication session is a video conference. 
     
     
         5 . The method of  claim 1 , wherein providing the generated data comprises providing the generated data for presentation, by one or more of the multiple endpoint devices, with the one or more media streams. 
     
     
         6 . The method of  claim 5 , wherein providing the generated data comprises providing the data over a communication network to the multiple endpoint devices. 
     
     
         7 . The method of  claim 1 , wherein the one or more media streams include audio data; and
 wherein determining the speaking time duration comprises processing the audio data to detect speaking time durations for different participants in the communication session.   
     
     
         8 . The method of  claim 1 , wherein providing the generated data comprises integrating the one or more indicators with a media stream of the communication session or a representation for a participant in the communication session. 
     
     
         9 . The method of  claim 1 , wherein determining a speaking time duration comprises monitoring speaking time durations of each of multiple participants in the communication session; and
 wherein generating the output data comprises generating indicators of respective levels of participation of the multiple participants based on speaking time durations of the multiple participants; and   wherein providing the output data comprises providing the indicators of the respective levels of participation of the multiple participants.   
     
     
         10 . The method of  claim 9 , comprising providing updated indicators of the respective levels of participation of the multiple participants as the communication session proceeds. 
     
     
         11 . A method performed by one or more computers, the method comprising:
 receiving, by one or more computers, one or more media streams of a communication session involving multiple endpoint devices, wherein the one or more media streams include audio data for speech by participants in the communication session, and wherein the one or more media streams are received during the communication session;   obtaining, by the one or more computers, speech recognition results for the one or more media streams of the communication session, the speech recognition results including recognized speech of the participants in the communication session;   generating, by the one or more computers, data comprising one or more indicators of levels of participation of the one or more participants in the communication session based on content of the recognized speech; and   providing, by the one or more computers, the generated data comprising the one or more indicators for presentation during the communication session.   
     
     
         12 . The method of  claim 11 , selecting a management action for the communication session from among a plurality of different management actions, wherein the selected management action is configured to alter transmission or presentation of the one or more media streams for one or more of the endpoint devices; and
 carrying out the selected management action or providing a recommendation for the selected management action.   
     
     
         13 . The method of  claim 1 , wherein providing the generated data comprises providing the generated data to at least one endpoint device involved in the communication session over a communication network for presentation to the at least one endpoint device with the one or more media streams. 
     
     
         14 . The method of  claim 11 , comprising evaluating, by the one or more computers, the speech recognition results to detect keywords that correspond to different emotional states. 
     
     
         15 . The method of  claim 11 , comprising detecting, based on audio data of the one or more media streams, utterance of one or more keywords from a set of keywords;
 wherein the one or more indicators are based on detecting utterance of the one or more keywords.   
     
     
         16 . A method performed by one or more computers, the method comprising:
 receiving, by one or more computers, one or more media streams of a communication session involving multiple endpoint devices, wherein the one or more media streams include audio data for speech by participants in the communication session, and wherein the one or more media streams are received during the communication session;   performing, by the one or more computers, analysis of intonation of speech described by the audio data in the one or more media streams of the communication session;   generating, by the one or more computers, data comprising one or more indicators of levels of participation of the one or more participants in the communication session based on the analysis of the intonation of the speech described by the audio data; and   providing, by the one or more computers, the generated data comprising the one or more indicators for presentation during the communication session.   
     
     
         17 . The method of  claim 16 , selecting a management action for the communication session from among a plurality of different management actions, wherein the selected management action is configured to alter transmission or presentation of the one or more media streams for one or more of the endpoint devices; and
 carrying out the selected management action or providing a recommendation for the selected management action.   
     
     
         18 . The method of  claim 16 , wherein providing the generated data comprises providing the generated data to at least one endpoint device involved in the communication session over a communication network for presentation to the at least one endpoint device with the one or more media streams. 
     
     
         19 . The method of  claim 16 , comprising determining a score for an emotional state of a participant based on the analysis of intonation of speech of the participant;
 wherein the one or more indicators are based on the determined score for the emotional state.   
     
     
         20 . The method of  claim 16 , comprising:
 determining a pattern of intonation of speech for a participant in the communication session; and   comparing the determined pattern of intonation with a reference pattern of intonation;   wherein the one or more indicators are based on the comparison.