Multi-modal information retrieval system

Abstract

Multi-modal system which enables documents to be interpreted as either or both of voice browser-based documents and/or visual browser-based documents for a thin client such as a portable telephone. Special techniques and additions are added into the document to enable the document to be converted between voice markup and visual markup languages. In addition, the document can be simultaneously viewed in both of the voice markup and the visual markup languages. Special techniques are used to allow keeping track of the browsing position within this document.

1 . A method, comprising: 
 receiving a document containing a first content portion and a second content portion;    providing a speech mark-up language version of said first content portion; and    providing a visual mark-up language version of said second content portion.    
   
   
       2 . The method of  claim 1  further including contemporaneously providing a visual mark-up language version of said first content portion.  
   
   
       3 . The method of  claim 1  further including contemporaneously providing a speech mark-up language version of said second content portion.  
   
   
       4 . The method of  claim 1  wherein the document includes a first tag associated with said first content portion.  
   
   
       5 . The method of  claim 4  wherein the document includes a second tag associated with said second content potion.  
   
   
       6 . The method of  claim 4  wherein said providing said speech mark-up language version of said first content portion includes interpreting said first tag.  
   
   
       7 . The method of  claim 6  wherein said providing said visual mark-up language version of said second content portion includes interpreting said second tag.  
   
   
       8 . The method of  claim 1  wherein the document includes a switch tag for facilitating switching from said speech mark-up language version to said visual mark-up language version.  
   
   
       9 . The method of  claim 1  wherein the document includes a switch tag for facilitating switching from said visual mark-up language version to said speech mark-up language version.  
   
   
       10 . A method, comprising: 
 browsing a document through either a speech-based mode or a visually-based mode;    tracking a current position of browsing of the document in either the speech-based mode or the visually-based mode; and    switching from browsing of the document in one of the speech-based or visually-based modes to the other of the modes.    
   
   
       11 . The method of  claim 10  wherein in the switching is performed in response to receipt of a command from a user.  
   
   
       12 . The method of  claim 10  wherein the switching comprises a page-based switching pursuant to which the browsing is continued from a predefined location on a page of the document associated with the current position.  
   
   
       13 . The method of  claim 10  wherein the switching comprises a link-based switching pursuant to which the browsing is continued at a predetermined location on a page of the document specified by a link selected to effect the switching.  
   
   
       15 . The method of  claim 10  wherein the switching is from the speech-based mode to the visually-based mode, the method further including providing a visual representation of content previously browsed during browsing of the document in the speech-based mode.  
   
   
       16 . The method of  claim 10  wherein the switching further includes providing visual interfaces to enable switching to the visually-based mode during browsing in the speech-based mode.  
   
   
       17 . The method of  claim 10  wherein the switching comprises a subsection-based switching enabling a first portion of a page of the document to be browsed in the speech-based mode and a second portion of a page of the document to be browsed in the visually-based mode.  
   
   
       18 . The method of  claim 1  further including automatically converting visual information upon which at least a portion of the document is based into speech-based subsections capable of being played during browsing of the document in the speech-based mode.  
   
   
       19 . A system, comprising: 
 an information storage unit which stores a document in a structured language, the document containing a first portion and a second portion;    a conversion server which interprets the document to enable the first portion to be provided in a speech mark-up language and the second portion to be provided in a visual mark-up language.    
   
   
       20 . The system of  claim 19  wherein the conversion server includes a retrieval module configured to retrieve Web content from a Web server and convert the Web content into the structured language.  
   
   
       21 . The system of  claim 20  wherein the retrieval module retrieves a main page included within the Web content and one or more branch pages accessible from the main page.  
   
   
       22 . The system of  claim 21  further including a prefetch cache associated with the conversion server disposed to store the structured language information corresponding to the main page and the one or more branch pages.