Systems and methods for collaborative note-taking

Abstract

Techniques are provided for determining collaborative notes and automatically recognizing speech, handwriting and other type of information. Domain and optional actor/speaker information associated with the support information is determined. An initial automatic speech recognition model is determined based on the domain and/or actor information. The domain and/or actor/speaker language model is used to recognize text in the speech information associated with the support information. Presentation support information such as slides, speaker notes and the like are determined. The semantic overlap between the support information and the salient non-function words in the recognized text and collaborative user feedback information are used to determine relevancy scores for the recognized text. Grammaticality, well formedness, self referential integrity and other features are used to determine correctness scores. Suggested collaborative notes are displayed in the user interface based on the salient non-function words. User actions in the user interface determine feedback signals. Recognition models such as automatic speech recognition, handwriting recognition are determined based on the feedback signals and the correctness and relevance scores.

1 . A method for displaying collaborative user notes comprising the steps of: determining portions of a first information channel; dynamically determining domain information based on the determined portions; determining context information temporally associated with the determined portions; determining a suggested note based on collaborative user feedback information and at least one of: the determined portions of the first information channel and the temporally related context information. 
   
   
       2 . The method of  claim 1 , in which portions of the first information channel contain ambiguities. 
   
   
       3 . The method of  claim 1 , in which a recognition model provides alternate salient tokens for recognized portions of the first information channel. 
   
   
       4 . The method of  claim 1 , wherein determining domain information comprises determining salient tokens based on at least one of: the first information channel, the context information channel and the at least one recognition model. 
   
   
       5 . The method of  claim 2 , further comprising the steps of: determining at least one of a relevance score and a correctness score between alternate portions of each ambiguous portion of the first information channel and associated portions of the context information channel; and determining an ordering of alternate portions based on at least one of the relevance score and the correctness score. 
   
   
       6 . The method of  claim 2 , further comprising the steps of: determining at least one of a relevance score and a correctness score between alternate hypotheses for each portion of the first information channel and associated portions of the context information channel; and determining an ordering of alternate hypotheses based on at least one of the relevance score and the correctness score. 
   
   
       7 . The method of  claim 5 , wherein determining the relevance score comprises determining a relatedness metric between portions of the first information channel and the context information channel. 
   
   
       8 . The method of  claim 7 , wherein the relatedness metric is at least one of a literal similarity metric and conceptual similarity metric. 
   
   
       9 . The method of  claim 8 , wherein the conceptual similarity is based on a lexicon. 
   
   
       10 . The method of  claim 9 , wherein the lexicon is the WordNet lexicon. 
   
   
       11 . The method of  claim 6 , wherein the correctness score is determined based on a correctness metric based on a well-formedness, grammaticality, semantics, syntax, discourse structure and discourse analysis. 
   
   
       12 . The method of  claim 6 , wherein the correctness score is based on collaborative user feedback. 
   
   
       13 . The method of  claim 6 , wherein the correctness score is based on an intrinsic probability that the ambiguous portions of the first information channel and the context information are disjoined. 
   
   
       14 . The method of  claim 6 , wherein the correctness score is based on phonetic overlap. 
   
   
       15 . The method of  claim 6 , wherein the correctness score is based on a predicted flow through the context information. 
   
   
       16 . The method of  claim 6 , wherein the correctness score is based on the presence of acronyms and associated expansions in the first information and the context information. 
   
   
       17 . The method of  claim 6 , wherein the context information is comprised of user notes and the correctness information is based on at least one of: semantic and lexical distance from the user notes. 
   
   
       18 . A system for displaying collaborative user notes comprising: a memory; an input/output circuit for retrieving portions of information from a first information channel and portions of context information from a second information channel and wherein the portions of information from the first information channel are temporally associated with portions of context information from the second information channel; a processor that dynamically determines domain information associated with the portions of information from the first information channel based on the portions of information from the first information channel and the temporally associated portions of context information from the second information channel and which determines a suggested note based on a collaborative user feedback signal ad at least one of: the determined portions of the first information channel and the temporally associated portions of context information from the second information channel. 
   
   
       19 . The system of  claim 18 , in which portions of the first information channel contain ambiguities. 
   
   
       20 . The system of  claim 18 , in which a recognition model provides alternate salient tokens for recognized portions of the first information channel. 
   
   
       21 . The system of  claim 18 , wherein determining domain information comprises determining salient tokens based on at least one of: the first information channel, the context information channel and the at least one recognition model. 
   
   
       22 . The system of  claim 19 , further comprising the steps of: determining at least one of a relevance score and a correctness score between alternate portions of each ambiguous portion of the first information channel and associated portions of the context information channel; and determining an ordering of alternate portions based on at least one of the relevance score and the correctness score. 
   
   
       23 . The system of  claim 19 , further comprising the steps of: determining at least one of a relevance score and a correctness score between alternate hypotheses for each portion of the first information channel and associated portions of the context information channel; and determining an ordering of alternate hypotheses based on at least one of the relevance score and the correctness score. 
   
   
       24 . The system of  claim 22 , wherein determining the relevance score comprises determining a relatedness metric between portions of the first information channel and the context information channel. 
   
   
       25 . The system of  claim 24 , wherein the relatedness metric is at least one of a literal similarity metric and conceptual similarity metric. 
   
   
       26 . The system of  claim 25 , wherein the conceptual similarity is based on a lexicon. 
   
   
       27 . The system of  claim 26 , wherein the lexicon is the WordNet® lexicon. 
   
   
       28 . The system of  claim 22 , wherein the correctness score is determined based on a correctness metric based on a well-formedness, grammaticality, semantics, syntax, discourse structure and discourse analysis. 
   
   
       29 . The system of  claim 22 , wherein the correctness score is based on collaborative user feedback. 
   
   
       30 . The system of  claim 22 , wherein the correctness score is based on an the intrinsic probability that the ambiguous portions of the first information channel and the context information are disjoined. 
   
   
       31 . The system of  claim 22 , wherein the correctness score is based on phonetic overlap. 
   
   
       32 . The system of  claim 22 , wherein the correctness score is based on a predicted flow through the context information. 
   
   
       33 . The system of  claim 22 , wherein the correctness score is based on the presence of acronyms and associated expansions in the first information and the context information. 
   
   
       34 . The system of  claim 22 , wherein the context information is comprised of user notes and the correctness information is based on at least one of: semantic and lexical distance from the user notes. 
   
   
       35 . Computer readable storage medium comprising: computer readable program code embodied on the computer readable storage medium, the computer readable program code usable to program a computer to display collaborative user notes comprising the steps of: determining portions of a first information channel; dynamically determining domain information based on the determined portions; determining context information temporally associated with the determined portions; determining a suggested note based on collaborative user feedback information and at least one of: the determined portions of the first information channel and the temporally related context information. 
   
   
       36 . A carrier wave encoded to transmit a control program, useable to program a computer to display collaborative user notes, to a device for executing the program, the control program comprising: instructions for determining portions of a first information channel; instructions for dynamically determining domain information based on the determined portions; instructions for determining context information temporally associated with the determined portions; instructions for determining a suggested note based on collaborative user feedback information and at least one of: the determined portions of the first information channel and the temporally related context information.