System and method for hybrid processing in a natural language voice services environment

Abstract

A system and method for hybrid processing in a natural language voice services environment that includes a plurality of multi-modal devices may be provided. In particular, the hybrid processing may generally include the plurality of multi-modal devices cooperatively interpreting and processing one or more natural language utterances included in one or more multi-modal requests. For example, a virtual router may receive various messages that include encoded audio corresponding to a natural language utterance contained in a multi-modal interaction provided to one or more of the devices. The virtual router may then analyze the encoded audio to select a cleanest sample of the natural language utterance and communicate with one or more other devices in the environment to determine an intent of the multi-modal interaction. The virtual router may then coordinate resolving the multi-modal interaction based on the intent of the multi-modal interaction.

What is claimed is: 
     
         1 . A method for hybrid processing in a natural language voice services environment, comprising:
 detecting at least one multi-modal interaction at an electronic device, wherein the multi-modal interaction includes at least a natural language utterance;   communicating one or more messages containing information relating to the multi-modal interaction to a virtual router in communication with the electronic device, wherein the electronic device communicates the one or more messages to the virtual router through a messaging interface;   receiving one or more messages containing information relating to an intent of the multi-modal interaction at the electronic device, wherein the electronic device receives the one or more messages from the virtual router through the messaging interface; and   resolving the multi-modal interaction at the electronic device based on the information contained in the one or more messages received from the virtual router.   
     
     
         2 . The method of  claim 1 , wherein the virtual router communicates with one or more additional electronic devices to determine the intent of the multi-modal interaction. 
     
     
         3 . The method of  claim 2 , wherein the virtual router determines a context for the multi-modal interaction and communicates with the one or more additional electronic devices in response to the determined context. 
     
     
         4 . The method of  claim 1 , wherein the virtual router communicates with a plurality of additional electronic devices to determine the intent of the multi-modal interaction. 
     
     
         5 . The method of  claim 1 , wherein the one or more messages communicated to the virtual router include encoded audio corresponding to the natural language utterance. 
     
     
         6 . The method of  claim 5 , wherein the virtual router communicates one or more messages containing the encoded audio to one or more additional electronic devices to determine the intent of the multi-modal interaction. 
     
     
         7 . The method of  claim 1 , wherein resolving the multi-modal interaction at the electronic device includes executing at least one request at the electronic device based on the intent of the multi-modal device interaction. 
     
     
         8 . The method of  claim 1 , wherein the multi-modal interaction further includes an additional non-voice interaction with the electronic device that relates to the natural language utterance. 
     
     
         9 . The method of  claim 8 , further comprising:
 establishing one or more device listeners on the electronic device, wherein the device listeners are configured to detect the natural language utterance and the additional non-voice interaction that relates to the natural language utterance; and   aligning timing information relating to the additional non-voice interaction and the natural language utterance.   
     
     
         10 . An electronic device for hybrid processing in a natural language voice services environment, wherein the electronic device is configured to:
 detect at least one multi-modal interaction that includes at least a natural language utterance;   communicate one or more messages containing information relating to the multi-modal interaction to a virtual router in communication with the electronic device, wherein the one or more messages are communicated to the virtual router through a messaging interface;   receive one or more messages containing information relating to an intent of the multi-modal interaction from the virtual router through the messaging interface; and   resolve the multi-modal interaction at the electronic device based on the information contained in the one or more messages received from the virtual router.   
     
     
         11 . The electronic device of  claim 10 , wherein the virtual router communicates with one or more additional electronic devices to determine the intent of the multi-modal interaction. 
     
     
         12 . The electronic device of  claim 11 , wherein the virtual router determines a context for the multi-modal interaction and communicates with the one or more additional electronic devices in response to the determined context. 
     
     
         13 . The electronic device of  claim 10 , wherein the virtual router communicates with a plurality of additional electronic devices to determine the intent of the multi-modal interaction. 
     
     
         14 . The electronic device of  claim 10 , wherein the one or more messages communicated to the virtual router include encoded audio corresponding to the natural language utterance. 
     
     
         15 . The electronic device of  claim 14 , wherein the virtual router communicates one or more messages containing the encoded audio to one or more additional electronic devices to determine the intent of the multi-modal interaction. 
     
     
         16 . The electronic device of  claim 10 , wherein the electronic device is further configured to execute at least one request at the electronic device based on the intent of the multi-modal device interaction to resolve the multi-modal interaction. 
     
     
         17 . The electronic device of  claim 10 , wherein the multi-modal interaction further includes an additional non-voice interaction with the electronic device that relates to the natural language utterance. 
     
     
         18 . The electronic device of  claim 17 , wherein the electronic device is further configured to:
 establish one or more device listeners configured to detect the natural language utterance and the additional non-voice interaction that relates to the natural language utterance; and   align timing information relating to the additional non-voice interaction and the natural language utterance.   
     
     
         19 . A virtual router for hybrid processing in a natural language voice services environment, wherein the virtual router is configured to:
 receive a plurality of messages that include encoded audio corresponding to a natural language utterance contained in a multi-modal interaction with a plurality of respective electronic devices;   analyze the encoded audio in the plurality of messages to determine one of the plurality of messages that provides a cleanest sample of the natural language utterance;   communicate one or more messages containing the encoded audio that provides the cleanest sample to a server in communication with the virtual router, wherein the one or more messages are communicated to the server through a messaging interface;   receive one or more messages containing information relating to an intent of the multi-modal interaction from the server through the messaging interface; and   return one or more messages containing the information relating to the intent of the multi-modal interaction to one or more of the plurality of electronic devices, wherein the one or more of the electronic devices resolve the multi-modal interaction based on the information relating to the intent of the multi-modal interaction.   
     
     
         20 . The virtual router of  claim 19 , wherein the virtual router is further configured to communicate with one or more of the plurality of electronic devices to determine the intent of the multi-modal interaction. 
     
     
         21 . The virtual router of  claim 20 , wherein the virtual router is further configured to determine a context for the multi-modal interaction and communicate with the one or more of the plurality of electronic devices in response to the determined context. 
     
     
         22 . The virtual router of  claim 19 , wherein the virtual router is further configured to communicate with more than one of the plurality of electronic devices to determine the intent of the multi-modal interaction.