Time travel search using digital content snapshots

Abstract

The present disclosure relates to systems, non-transitory computer-readable media, and methods for capturing snapshots of digital content displayed on a client device and searching through the captured content. The disclosed systems provide a search function for effectively traveling back in time to identify digital content previously displayed on a client device. The disclosed systems provide options for capturing snapshots of content displayed on a display screen, extracting data from the snapshots, and storing the snapshots for use when populating search results. The disclosed systems utilize machine learning models to extract text and/or to generate text versions of snapshots including extracted text, descriptions of images, transcripts of videos, and/or textual summaries from displayed documents or webpages. In response to a search query, the disclosed systems can produce search results that include digital videos including captured snapshots of content displayed by a client device over time.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 receiving, from a client device, a search query defining query content;   identifying, from the client device, digital content being displayed on the client device throughout navigation of at least one computer application;   extracting text from the digital content being displayed on the client device;   determining, for the extracted text, a confidence score by performing a semantic comparison of the search query and the extracted text; and   generating, for display on the client device and based on the confidence score, search results for the search query, the search results comprising the extracted text and indicating a content item associated with the extracted text.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the digital content being displayed on the client device comprises digital content not stored by a user account associated with the client device. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein extracting text from the digital content comprises extracting text from the digital content using a large language model. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein extracting text from the digital content comprises generating a description of an image being displayed on the client device, a transcript of a video being displayed on the client device, or a textual summary of a document or interface being displayed on the client device. 
     
     
         5 . The computer-implemented method of  claim 1 , wherein generating search results based on the confidence score comprises generating search results based on a threshold similarity between the extracted text and the search query. 
     
     
         6 . The computer-implemented method of  claim 1 , further comprising storing application data associated with the digital content being displayed on the client device. 
     
     
         7 . The computer-implemented method of  claim 6 , wherein the application data comprises an application or location for the digital content being displayed on the client device. 
     
     
         8 . A system comprising:
 one or more processors; and   a memory coupled to the one or more processors, wherein the memory includes instructions executable by the one or more processors to:
 identify digital content being displayed on a client device throughout navigation of at least one computer application; 
 extract data from the digital content being displayed on the client device; 
 receive, from the client device, a search query defining query content; 
 determine, for the extracted data, a confidence score by performing a semantic comparison of the search query and the extracted data; and 
 generate, for display on the client device and based on the confidence score, search results for the search query, the search results indicating at least one content item associated with the extracted data. 
   
     
     
         9 . The system of  claim 8 , wherein the memory further includes instructions executable by the one or more processors to extract data from the digital content being displayed on the client device by extracting a topic from the digital content being displayed on the client device. 
     
     
         10 . The system of  claim 8 , wherein the memory further includes instructions executable by the one or more processors to extract data from the digital content being displayed on the client device by extracting selectable content from the digital content being displayed on the client device. 
     
     
         11 . The system of  claim 8 , wherein the search query comprises a timeframe parameter, and wherein the extracted data comprises a timestamp associated with the digital content being displayed on the client device. 
     
     
         12 . The system of  claim 8 , wherein the memory further includes instructions executable by the one or more processors to identify the digital content by identifying digital content being displayed across multiple computer applications running on the client device during a login session. 
     
     
         13 . The system of  claim 12 , wherein the memory further includes instructions executable by the one or more processors to extract data from the digital content being displayed across multiple computer applications running on the client device during the login session. 
     
     
         14 . The system of  claim 8 , wherein the memory further includes instructions executable by the one or more processors to:
 determining that a digital content item being displayed on the client device is associated with a computer application or content type designated as impermissible; and   excluding the digital content item from extracting data.   
     
     
         15 . A non-transitory computer readable medium storing instructions which, when executed by at least one processor, cause the at least one processor to:
 receive, from a client device, a search query defining query content;   perform a semantic comparison of the search query and extracted data from digital content previously displayed on the client device;   determine a confidence score based on the semantic comparison; and   generate, for display on the client device and based on the confidence score, search results for the search query, the search results indicating at least one content item associated with the extracted data.   
     
     
         16 . The non-transitory computer readable medium of  claim 15 , wherein the search results comprise digital content items. 
     
     
         17 . The non-transitory computer readable medium of  claim 16 , further storing instructions which, when executed by at least one processor, cause the at least one processor to, upon receiving an indication of a selection of a digital content item from the search results, open the digital content item within an associated application. 
     
     
         18 . The non-transitory computer readable medium of  claim 15 , wherein the extracted data comprises extracted text, an extracted topic, or an extracted summary. 
     
     
         19 . The non-transitory computer readable medium of  claim 15 , further storing instructions which, when executed by at least one processor, cause the at least one processor to generate the search results by searching the extracted data for content corresponding to the search query. 
     
     
         20 . The non-transitory computer readable medium of  claim 15 , wherein the digital content previously displayed on the client device comprises a webpage, and the extracted data comprises a uniform resource locator (URL).