Creating and Using a Graph of Organisation-Specific Knowledge From Conversation Data

Abstract

A computer-implemented method of building a graph of organisation-specific knowledge includes the step of receiving conversation data. Words and phrases within the conversation data that describe organisational knowledge are detected. Sets of candidate knowledge tuples are extracted. Each tuple includes a sequence of elements that represent respective words, phrases, or sentences that describe an item of knowledge. The elements in each tuple are ordered according to an ontology of the item of knowledge. The tuples in each set describe respective versions of the item of organisational knowledge. Exemplar tuples that correspond to candidate tuples in respective sets that contain elements that occur more than a threshold number of times in the text are created. The graph is initialised from nodes representing the elements of the exemplar tuples. Edges represent relationships between the elements. Sets of nodes that represent the same element of organisational knowledge are progressively determined and merged into exemplary nodes.

1 . A computer-implemented method of building a graph of organisation-specific knowledge, the method comprising the steps of:
 receiving conversation data representing at least one conversation;   detecting words and phrases within the conversation data that describe organisational knowledge;   extracting sets of candidate knowledge tuples from the conversation data, each tuple including a sequence of elements that represent respective words, phrases, or sentences that together describe an item of organisational knowledge, the elements in each tuple being ordered according to an ontology of the item of organisational knowledge, and the tuples in each set describing respective versions of the item of organisational knowledge;   creating exemplar tuples that correspond to candidate tuples in respective sets that contain elements that occur more than a threshold number of times in the text;   initialising the graph from nodes representing the elements of the exemplar tuples, and edges representing relationships between the elements; and   progressively determining and merging sets of nodes that represent the same element of organisational knowledge into exemplary nodes of the graph.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the step of detecting words and phrases within the text that describe the items of organisational knowledge includes the step of using natural language processing to identify the words, phrases or sentences. 
     
     
         3 . The computer-implemented method of  claim 2 , wherein the different types of organisational knowledge may include one or more of the following:
 programmatic messages inserted into conversations, along with their type such as on hold messages, IVR responses, legal disclaimers, terms and conditions, etc.;   products or services provided by the organisation;   organisational structure such as departments, roles, and personnel; and   other organisational knowledge that provides context to conversations, such as systems, processes, industry details, partner organisations, etc.   
     
     
         4 . The computer-implemented method of  claim 2 , wherein the step of detecting words and phrases within the text includes the step of using a neural network language model. 
     
     
         5 . The computer-implemented method of  claim 4 , wherein the step of using a neural network language model includes the step of using a supervised classifier that is trained to discriminate between sentences containing organisational knowledge and normal conversational speech sentences by using an annotated set of sentences containing known organisational knowledge as well as normal conversational speech. 
     
     
         6 . The computer-implemented method of  claim 4 , which includes the step of normalising text representing the detected words and phrases to facilitate upstream comparisons. 
     
     
         7 . The computer-implemented method of  claim 1 , wherein the ontology of each item of organisational knowledge is hierarchical such that elements of each candidate tuple are sequenced in hierarchical order with each candidate tuple having the form {“Level 1”, “Level 2”; “Level 3”; . . . , “Level N”}, where Level 1 is a broadest category of the item of organisational information, and Level N is a narrowest category. 
     
     
         8 . The computer-implemented method of  claim 7 , wherein the step of initialising the graph includes the steps of:
 a) for each candidate tuple from a counted set of candidate tuples having an occurrence that is greater than the threshold number, creating a new Level 1 node for each unique Level 1 field;   b) for each candidate tuple with a given Level 1 field, creating a Level 2 node connected to a corresponding Level 1 node for each unique Level 2 field;   c) for each candidate tuple with a given Level 1 and Level 2 field combination, creating a Level 3 node connected to a corresponding Level 2 node for each unique Level 3 field;   d) repeating steps (a) to (c) until all tuple levels have been processed; and   e) for each node, storing the corresponding count from the counted set as metadata.   
     
     
         9 . The computer-implemented method of  claim 1 , wherein the nodes that represent the same element of organisational knowledge are associated with the exemplary node as metadata to facilitate matching in subsequent detection processes and are subsequently pruned from the graph to reduce its complexity. 
     
     
         10 . The computer-implemented method of  claim 1 , wherein a determination of whether any two nodes represent the same element of organisational knowledge is carried out by transforming the associated candidate tuples into vectors in an embedding space and measuring a distance between the two vectors or measuring cosine similarity between the two vectors. 
     
     
         11 . The computer-implemented method of  claim 1 , wherein a determination of whether any two nodes represent the same element of organisational knowledge is carried out by deducing similarity using neural network language models. 
     
     
         12 . The computer-implemented method of  claim 1 , wherein a model of the graph is trained by:
 a) determining an occurrence count threshold for which it is assumed that elements of organisational knowledge above this count are good quality exemplars; and   b) merging lower occurrence elements into the higher occurrence elements if predetermined similarity conditions are met.   
     
     
         13 . The computer-implemented method of  claim 1 , wherein a model is trained by:
 a) determining a similarity threshold; and   b) merging the most similar elements above a predetermined occurrence threshold until no possible mergers lie above the similarity threshold.   
     
     
         14 . A method of determining organisational knowledge using a comprising the steps of:
 building a graph of organisation-specific knowledge that includes the steps of:
 receiving conversation data representing at least one conversation; 
 detecting words and phrases within the conversation data that describe organisational knowledge; 
 extracting sets of candidate knowledge tuples from the conversation data, each tuple including a sequence of elements that represent respective words, phrases, or sentences that together describe an item of organisational knowledge, the elements in each tuple being ordered according to an ontology of the item of organisational knowledge, and the tuples in each set describing respective versions of the item of organisational knowledge; 
 creating exemplar tuples that correspond to candidate tuples in respective sets that contain elements that occur more than a threshold number of times in the text; 
 initialising the graph from nodes representing the elements of the exemplar tuples, and edges representing relationships between the elements; and 
 progressively determining and merging sets of nodes that represent the same element of organisational knowledge into exemplary nodes of the graph; 
   providing a transcript representing the at least one conversation;   detecting elements of organisational knowledge in the transcript by performing a matching function to measure a similarity between the elements of organisational knowledge in the transcript and the elements of organisational knowledge represented by the nodes of the graph; and   outputting detected elements of organisational knowledge as metadata associated with the at least one conversation.   
     
     
         15 . The method of  claim 14 , wherein the step of performing the matching function includes the step of computing a similarity between TF-IDF vector representations of test elements of organisational knowledge from the transcript and exemplar elements of organisational knowledge from the graph. 
     
     
         16 . The method of  claim 15 , wherein the step of computing the similarity between the TF-IDF vector representations includes the steps of:
 a) using an NLP tokenizer or embedding network to convert the transcript and exemplar elements of organisational knowledge into vectors of length M, representing a number of possible tokens;   b) converting N exemplar elements of organisational knowledge into numerical TF-IDF form to generate an N×M matrix;   c) converting T transcription elements of organisational knowledge into numerical TF-IDF form to generate a T×M matrix; and   d) computing the similarity between the matrices.   
     
     
         17 . The method of  claim 16 , wherein the step of computing the similarity between the matrices includes the step of computing a cosine similarity between all combinations of transcript and exemplar elements of organisational knowledge. 
     
     
         18 . The method of  claim 16 , wherein the step of computing the similarity between the matrices is carried out using sparse matrix representations. 
     
     
         19 . A system for building a graph of organisation-specific knowledge, the system comprising:
 a non-transitory computer-readable medium with instructions encoded thereon; and   one or more processors configured to, when executing the instructions, perform operations of:
 receiving conversation data representing at least one conversation; 
 detecting words and phrases within the conversation data that describe organisational knowledge;
 extracting sets of candidate knowledge tuples from the conversation data, each tuple including a sequence of elements that represent respective words and phrases that together describe an item of organisational knowledge, the elements in each tuple being ordered according to an ontology of the item of organisational knowledge, and the tuples in each set describing respective versions of the item of organisational knowledge; 
 creating exemplar tuples that correspond to candidate tuples in respective sets that contain elements that occur more than a threshold number of times in the text; 
 initialising the graph from nodes representing the elements of the exemplar tuples; and 
 progressively determining and merging sets of nodes that represent the same element of organisational knowledge into exemplary nodes of the graph. 
 
   
     
     
         20 . A computer program product comprising a non-transitory computer-readable storage medium containing computer program code, the computer program code, when executed by one or more processors causing the one or more processors to perform operations, the computer program code comprising instructions to:
 receive conversation data representing at least one conversation;   detect words and phrases within the conversation data that describe organisational knowledge;   extract sets of candidate knowledge tuples from the conversation data, each tuple including a sequence of elements that represent respective words and phrases that together describe an item of organisational knowledge, the elements in each tuple being ordered according to an ontology of the item of organisational knowledge, and the tuples in each set describing respective versions of the item of organisational knowledge;   create exemplar tuples that correspond to candidate tuples in respective sets that contain elements that occur more than a threshold number of times in the text;   initialise the graph from nodes representing the elements of the exemplar tuples; and   progressively determine and merge sets of nodes that represent the same element of organisational knowledge into exemplary nodes of the graph.