Pinning artifacts for expansion of search keys and search spaces in a natural language understanding (nlu) framework

Abstract

Present embodiments include an agent automation framework having an artifact pinning subsystem that pins meaning representations of a search space to enable the agent automation system to target particularly relevant candidates for improved inferences. To generate the search space, the artifact pinning subsystem may determine multiple understandings of sample utterances within intent-entity models to generate meaning representations. The sample utterances generally each belong to an identified intent that may have been labeled with a particular entity, within a structure defined by the intent-entity models. To validate the relevance of each meaning representation for an identified intent, the artifact pinning subsystem may pin meaning representations that include the particular intent and include a respective entity corresponding to the labeled entity. In addition to model-based entity pinning, the search space may also be generated with respect to a contextual intent of an on-going conversation between a user and a behavior engine.

What is claimed is: 
     
         1 . An agent automation system, comprising:
 a memory configured to store a natural language understanding (NLU) framework including a meaning search subsystem; and   a processor configured to execute instructions to cause the meaning search subsystem of the NLU framework to perform actions comprising:
 receiving a plurality of sample meaning representations, wherein one or more labeled meaning representations of the plurality of sample meaning representations comprise a particular intent and a labeled entity that is associated with the particular intent; 
 for each labeled meaning representation of the one or more labeled meaning representations, pinning a set of meaning representations from the plurality of sample meaning representations, wherein each meaning representation of the set of meaning representations comprises the particular intent and a respective entity corresponding to the labeled entity of the labeled meaning representation; and 
 generating a search space based at least in part on each set of meaning representations corresponding to each labeled meaning representation, wherein the meaning search subsystem is configured to compare a search key meaning representation to search space meaning representations to identify one or more of the search space meaning representations as matches for the search key meaning representation. 
   
     
     
         2 . The agent automation system of  claim 1 , wherein the instructions are configured to cause the meaning search subsystem to perform actions comprising:
 receiving a second plurality of sample meaning representations comprising one or more second labeled meaning representations, wherein one or more second labeled meaning representations of the second plurality of sample meaning representations comprise a particular artifact and a labeled artifact that is associated with the particular artifact; and   for each second labeled meaning representation of the second plurality of sample meaning representations, pinning a second set of meaning representations from the second plurality of sample meaning representations, wherein each meaning representation of the second set of meaning representations comprises the particular artifact and a respective artifact corresponding to the labeled artifact of the second labeled meaning representation; and
 generating the search space based at least in part on each set of meaning representations corresponding to each labeled meaning representation and each second set of meaning representations corresponding to each second labeled meaning representation. 
   
     
     
         3 . The agent automation system of  claim 2 , wherein the particular artifact is a second particular intent, and wherein the labeled artifact is a second labeled entity that is associated with the second particular intent of the respective second labeled meaning representation. 
     
     
         4 . The agent automation system of  claim 2 , wherein the plurality of sample meaning representations and the second plurality of sample meaning representations are received from a meaning extraction subsystem of the NLU framework. 
     
     
         5 . The agent automation system of  claim 1 , wherein the plurality of sample meaning representations are received from a meaning extraction subsystem of the NLU framework, wherein the meaning extraction subsystem generated the plurality of sample meaning representations from an intent-entity model comprising a plurality of sample utterances, and wherein the plurality of sample utterances comprise a set of intents that is associated with a set of entities. 
     
     
         6 . The agent automation system of  claim 5 , wherein the labeled entity is associated with the particular intent by an author of the intent-entity model, and wherein a parameter indicative of the association is stored within the intent-entity model. 
     
     
         7 . The agent automation system of  claim 1 , wherein the instructions are configured to cause the meaning search subsystem to perform actions comprising:
 before generating the search space, for each labeled meaning representation of the one or more labeled meaning representations:
 discarding a remaining set of meaning representations of the plurality of sample meaning representations, wherein the remaining set of meaning representations do not comprise the particular intent and a respective entity corresponding to the labeled entity. 
   
     
     
         8 . The agent automation system of  claim 1 , wherein the instructions are configured to cause the meaning search subsystem to perform actions comprising:
 before generating the search space, re-expressing the set of meaning representations into a set of re-expressed meaning representations by rearranging, adding, removing, and/or transforming tokens represented by respective nodes of the set of meaning representations; and   disregarding any re-expressed meaning representations of the set of re-expressed meaning representations that do not respectively comprise the particular intent and the labeled entity.   
     
     
         9 . The agent automation system of  claim 8 , wherein the instructions are configured to cause the meaning search subsystem to perform actions comprising:
 removing intent-level duplicates, entity-level duplicates, or a combination thereof from the set of re-expressed meaning representations before generating the search space.   
     
     
         10 . The agent automation system of  claim 1 , wherein the instructions are configured to cause the meaning search subsystem to perform actions comprising:
 removing intent-level duplicates, entity-level duplicates, or a combination thereof from the plurality of sample meaning representations before identifying the set of meaning representations from the plurality of sample meaning representations.   
     
     
         11 . The agent automation system of  claim 1 , wherein the plurality of sample meaning representations each comprise respective nodes representing tokens of a respective sample utterance, wherein a first portion of the respective nodes of each labeled meaning representation are tagged with the particular intent, and wherein a second portion of the respective nodes of each labeled meaning representation are tagged with the labeled entity. 
     
     
         12 . The agent automation system of  claim 1 , wherein the instructions are configured to cause the meaning search subsystem to perform actions comprising:
 receiving the search key meaning representation; and   comparing the search key meaning representation to the search space meaning representations to identify the one or more search space meaning representations that are matches for the search key meaning representation.   
     
     
         13 . A method of operating an agent automation system, comprising:
 determining a contextual intent of a dialog between a user and a behavior engine of the agent automation system;   generating a meaning representation of a received user utterance of the dialog as a search key;   refining a search space that comprises meaning representations of sample utterances of an intent-entity model, wherein refining comprises selecting a portion of the meaning representations of the search space that include the contextual intent to yield a refined search space; and   comparing the search key to the refined search space to identify at least one matching meaning representation from the meaning representations of the refined search space.   
     
     
         14 . The method of  claim 13 , wherein determining the contextual intent comprises determining an intent that corresponds to a flow being executed by the behavior engine with respect to the dialog with the user. 
     
     
         15 . The method of  claim 13 , wherein the method comprises, before refining the search space, pruning irrelevant meaning representations from the search space that do not include the contextual intent. 
     
     
         16 . A non-transitory, computer-readable medium storing instructions that, when executed by one or more processors of an agent automation system, cause the agent automation system to implement a meaning search subsystem to:
 receive a plurality of meaning representations, wherein one or more labeled meaning representations of the plurality of meaning representations comprise a particular intent and a particular entity that is associated with the particular intent;   for each labeled meaning representation of the one or more labeled meaning representations, pin a set of meaning representations from the plurality of meaning representations, wherein each meaning representation of the set of meaning representations comprises the particular intent and the particular entity; and   generate a search space based at least in part on each set of meaning representations that corresponds to each labeled meaning representation.   
     
     
         17 . The non-transitory, computer-readable medium of  claim 16 , wherein the search space is based at least in part on each set of meaning representations that correspond to each labeled meaning representation. 
     
     
         18 . The non-transitory, computer-readable medium of  claim 17 , wherein the particular entity of each labeled meaning representation was associated with the particular intent of an intent-entity model, and wherein the intent-entity model is utilized to determine the plurality of meaning representations. 
     
     
         19 . The non-transitory, computer-readable medium of  claim 16 , wherein the instructions cause the agent automation system to implement the meaning search subsystem to:
 re-express the set of meaning representations into a set of re-expressed meaning representations by rearranging, adding, removing, and/or transforming tokens represented by respective nodes of the set of meaning representations; and   generate the search space based on each re-expressed meaning representation of the set of re-expressed meaning representations.   
     
     
         20 . The non-transitory, computer-readable medium of  claim 16 , wherein the instructions cause the agent automation system to implement the meaning search subsystem to disregard any re-expressed meaning representations of the set of re-expressed meaning representations that do not respectively comprise the particular intent and the particular entity, before generating the search space.