Data driven dialog management

Abstract

Techniques for optimizing a system to improve an overall user satisfaction in a speech controlled system are described. A user speaks an utterance and the system compares an expected sum of user satisfaction values for each action to make a decision as to how best to process the utterance. As a result, the system may make a decision that decreases user satisfaction in the short term but increases user satisfaction in the long term. The system may estimate a user satisfaction value and associate the estimated user satisfaction value with a current dialog state. By tracking user satisfaction values over time, the system may train machine learning models to optimize the expected sum of user satisfaction values. This improves how the system selects an action or application to which to dispatch the dialog state and how a specific application selects an action or intent corresponding to the command.

1 .- 20 . (canceled) 
     
     
         21 . A computer-implemented method, comprising:
 receiving first input data corresponding to a first natural language user input;   performing natural language understanding (NLU) processing on the first input data to determine first NLU result data;   processing the first NLU result data to determine an action responsive to the first natural language input;   performing the action to determine first output data;   causing output of the first output data;   receiving second input data corresponding to a second natural language user input; and   processing the second input data to determine user satisfaction data representing a likelihood a user was satisfied with the first output data.   
     
     
         22 . The computer-implemented method of  claim 21 , wherein performing the NLU processing comprises operation of a first component, and wherein the method further comprises:
 updating the first component based at least in part on the user satisfaction data.   
     
     
         23 . The computer-implemented method of  claim 21 , further comprising:
 determining second output data corresponding to a requested confirmation related to the action; and   causing output of the second output data,   wherein the second input data is in response to output of the second output data.   
     
     
         24 . The computer-implemented method of  claim 21 , further comprising:
 determining second user satisfaction data representing a potential user satisfaction corresponding to the action,   wherein determining the action is further based at least in part on the second user satisfaction data.   
     
     
         25 . The computer-implemented method of  claim 21 , wherein:
 performing the NLU processing comprises operation of a first component; and   processing the second input data to determine user satisfaction data comprises operation of a second component different from the first component.   
     
     
         26 . The computer-implemented method of  claim 21 , further comprising:
 determining dialog data based at least in part on the second input data,   wherein processing the second input data to determine user satisfaction data is based at least in part on the dialog data.   
     
     
         27 . The computer-implemented method of  claim 21 , further comprising:
 receiving image data corresponding to the second input data; and   performing computer vision processing using the image data to determine first data,   wherein determination of the user satisfaction data is further based at least in part on the first data.   
     
     
         28 . The computer-implemented method of  claim 21 , further comprising:
 determining the second input data comprises user feedback data.   
     
     
         29 . The computer-implemented method of  claim 21 , further comprising:
 determining the second input data corresponds to a same command as the first input data,   wherein the user satisfaction data is based at least in part on the second input data corresponding to the same command as the first input data.   
     
     
         30 . The computer-implemented method of  claim 21 , further comprising:
 determining history data corresponding to the first input data,   wherein the user satisfaction data is based at least in part on the history data.   
     
     
         31 . A system comprising:
 at least one processor; and   at least one memory comprising instructions that, when executed by the at least one processor, cause the system to:
 receive first input data corresponding to a first natural language user input; 
 perform natural language understanding (NLU) processing on the first input data to determine first NLU result data; 
 process the first NLU result data to determine an action responsive to the first natural language input; 
 perform the action to determine first output data; 
 cause output of the first output data; 
 receive second input data corresponding to a second natural language user input; and 
 process the second input data to determine user satisfaction data representing a likelihood a user was satisfied with the first output data. 
   
     
     
         32 . The system of  claim 31 , wherein performing the NLU processing comprises operation of a first component, and wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 update the first component based at least in part on the user satisfaction data.   
     
     
         33 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine second output data corresponding to a requested confirmation related to the action; and   cause output of the second output data,   wherein the second input data is in response to output of the second output data.   
     
     
         34 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine second user satisfaction data representing a potential user satisfaction corresponding to the action,   wherein determination of the action is further based at least in part on the second user satisfaction data.   
     
     
         35 . The system of  claim 31 , wherein:
 performing the NLU processing comprises operation of a first component; and   processing the second input data to determine user satisfaction data comprises operation of a second component different from the first component.   
     
     
         36 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine dialog data based at least in part on the second input data,   wherein the instructions that cause the system to process the second input data to determine user satisfaction data are based at least in part on the dialog data.   
     
     
         37 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 receive image data corresponding to the second input data; and   perform computer vision processing using the image data to determine first data,   wherein the instructions that cause the system to determine the user satisfaction data are further based at least in part on the first data.   
     
     
         38 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine the second input data comprises user feedback data.   
     
     
         39 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine the second input data corresponds to a same command as the first input data,   wherein the user satisfaction data is based at least in part on the second input data corresponding to the same command as the first input data.   
     
     
         40 . The system of  claim 31 , wherein the at least one memory further comprises instructions that, when executed by the at least one processor, further cause the system to:
 determine history data corresponding to the first input data,   wherein the user satisfaction data is based at least in part on the history data.