Exploration of User Memories in Multi-turn Dialogs for Assistant Systems

Abstract

In one embodiment, a method includes receiving a first user input from a user from a client system associated with the user, accessing a memory-data store comprising digital memories, wherein each digital memory comprises media content items, temporal information associated with the media content items, and relational information pointing to second digital memories, selecting priming digital memories from the digital memories based on the first user input, proactively selecting related digital memories based on the relational information associated with the priming digital memories, generating a priming response based on the priming digital memories and a proactive suggestion based on the related digital memories, and sending instructions for presenting the priming response and the proactive suggestion to the client system responsive the first user input.

What is claimed is: 
     
         1 . A method comprising, by one or more computing systems:
 receiving, from a client system associated with a user, a first user input from the user;   accessing a memory-data store comprising a plurality of digital memories, wherein each digital memory comprises one or more media content items, temporal information associated with the one or more media content items, and relational information pointing to one or more second digital memories;   selecting one or more priming digital memories from the plurality of digital memories based on the first user input;   proactively selecting one or more related digital memories based on the relational information associated with the priming digital memories;   generating a priming response based on the priming digital memories and a proactive suggestion based on the related digital memories; and   sending, to the client system responsive the first user input, instructions for presenting the priming response and the proactive suggestion.   
     
     
         2 . The method of  claim 1 , further comprising:
 generating a chit-chat response based on the priming digital memories and the related digital memories; and   sending, to the client system responsive the first user input, instructions for presenting the chit-chat response.   
     
     
         3 . The method of  claim 1 , wherein the memory-data store comprises a memory graph, and wherein the memory graph comprises a plurality of nodes and a plurality of edges connecting the nodes, wherein the plurality of nodes correspond to the plurality of digital memories, respectively, and wherein each edge connecting two nodes corresponds to relational information associated with the respective digital memories corresponding to the two nodes. 
     
     
         4 . The method of  claim 3 , further comprising:
 generating the memory graph based on a plurality of personal data associated with the user and a knowledge graph.   
     
     
         5 . The method of  claim 3 , further comprising:
 updating the memory graph based on user interactions with one or more of the priming digital memories or one or more of the related digital memories.   
     
     
         6 . The method of  claim 1 , wherein the one or more media content items of each digital memory comprise one or more of an image, a video clip, an email, a text, a file folder, an audio clip, or a post. 
     
     
         7 . The method of  claim 1 , wherein each digital memory further comprises one or more of social information associated with one or more of the media content items, visual information associated with one or more of the media content items, dialog information associated with one or more of the media content items, or location information associated with one or more of the media content items. 
     
     
         8 . The method of  claim 1 , further comprising:
 receiving, from the client system, a second user input from the user, wherein the related digital memories are selected responsive to the second user input.   
     
     
         9 . The method of  claim 1 , wherein proactively selecting the one or more related digital memories is further based on user context associated with the user. 
     
     
         10 . The method of  claim 1 , further comprising:
 receiving, from the client system, a second user input from the user, wherein the second user input references one or more of the priming or related digital memories; and   generating a response to the second user input based on one or more of the priming or related digital memories referenced in the second user input.   
     
     
         11 . The method of  claim 1 , further comprising:
 parsing the first user input to identify one or more priming memory slots, wherein selecting the one or more priming digital memories is based on the one or more priming memory slots.   
     
     
         12 . The method of  claim 1 , wherein the first user input references an ambiguous mention, wherein the method further comprises disambiguating the ambiguous mention referenced in the first user input. 
     
     
         13 . One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
 receive, from a client system associated with a user, a first user input from the user;   access a memory-data store comprising a plurality of digital memories, wherein each digital memory comprises one or more media content items, temporal information associated with the one or more media content items, and relational information pointing to one or more second digital memories;   select one or more priming digital memories from the plurality of digital memories based on the first user input;   proactively select one or more related digital memories based on the relational information associated with the priming digital memories;   generate a priming response based on the priming digital memories and a proactive suggestion based on the related digital memories; and   send, to the client system responsive the first user input, instructions for presenting the priming response and the proactive suggestion.   
     
     
         14 . The media of  claim 13 , wherein the software is further operable when executed to:
 generate a chit-chat response based on the priming digital memories and the related digital memories; and   send, to the client system responsive the first user input, instructions for presenting the chit-chat response.   
     
     
         15 . The media of  claim 13 , wherein the memory-data store comprises a memory graph, and wherein the memory graph comprises a plurality of nodes and a plurality of edges connecting the nodes, wherein the plurality of nodes correspond to the plurality of digital memories, respectively, and wherein each edge connecting two nodes corresponds to relational information associated with the respective digital memories corresponding to the two nodes. 
     
     
         16 . The media of  claim 13 , wherein each digital memory further comprises one or more of social information associated with one or more of the media content items, visual information associated with one or more of the media content items, dialog information associated with one or more of the media content items, or location information associated with one or more of the media content items. 
     
     
         17 . A system comprising: one or more processors; and a non-transitory memory coupled to the processors comprising instructions executable by the processors, the processors operable when executing the instructions to:
 receive, from a client system associated with a user, a first user input from the user;   access a memory-data store comprising a plurality of digital memories, wherein each digital memory comprises one or more media content items, temporal information associated with the one or more media content items, and relational information pointing to one or more second digital memories;   select one or more priming digital memories from the plurality of digital memories based on the first user input;   proactively select one or more related digital memories based on the relational information associated with the priming digital memories;   generate a priming response based on the priming digital memories and a proactive suggestion based on the related digital memories; and   send, to the client system responsive the first user input, instructions for presenting the priming response and the proactive suggestion.   
     
     
         18 . The system of  claim 17 , wherein the processors are further operable when executing the instructions to:
 generate a chit-chat response based on the priming digital memories and the related digital memories; and   send, to the client system responsive the first user input, instructions for presenting the chit-chat response.   
     
     
         19 . The system of  claim 17 , wherein the memory-data store comprises a memory graph, and wherein the memory graph comprises a plurality of nodes and a plurality of edges connecting the nodes, wherein the plurality of nodes correspond to the plurality of digital memories, respectively, and wherein each edge connecting two nodes corresponds to relational information associated with the respective digital memories corresponding to the two nodes. 
     
     
         20 . The system of  claim 17 , wherein each digital memory further comprises one or more of social information associated with one or more of the media content items, visual information associated with one or more of the media content items, dialog information associated with one or more of the media content items, or location information associated with one or more of the media content items.