Targeted search of backup data using facial recognition

Abstract

This application relates to targeted search of backup data. A data storage system can provide a targeted search of backup data using facial recognition. Upon receiving a search query that identifies a specific user whose profile is stored in a user database, the data storage system can retrieve a profile photograph of the specific user, perform facial recognition on photographs included in the backup data, and output a search result including a set of photographs in the backup data that match the facial features of the specific user.

1 . (canceled) 
     
     
         2 . A computer-implemented method for performing a targeted search on backup data, the method comprising:
 receiving a request to back up copies of primary data, wherein the primary data comprises a plurality of images wherein each image comprises metadata comprising at least one of date, time, location, size, creator, and file type, stored on one or more primary storage devices associated with a client computing device and wherein the client computing device is associated with a requesting user;   backing up the plurality of images associated with the requesting user;   storing, as backup copies, user profiles and profile photographs associated with the user profiles, wherein the user profiles comprise at least the requesting user's profile and its associated profile photograph, and images from users other than the requesting user;   storing, as backup copies, the backed-up plurality of images associated with the requesting user;   receiving, from the client computing device, a search query to search the stored backup copies;   identifying, based on the search query, a target user profile from the backup copies, wherein the target user profile is one of: the requesting user's profile and a user profile other than the requesting user's profile;   obtaining a target profile photograph associated with the target user profile, wherein the target profile photograph comprises one or more facial features;   performing facial recognition operations on the target profile photograph;   identifying at least one image among the backup copies of images that matches the one or more facial features from the target profile photograph; and   transmitting a search result comprising the identified at least one image to the client computing device in response to the search query.   
     
     
         3 . The method of  claim 2 , wherein the performing facial recognition operations comprises:
 detecting a region in the target profile photograph that corresponds to the target user's face; and   extracting one or more facial features from the detected region in the target profile photograph.   
     
     
         4 . The method of  claim 2 , wherein the backup copies are stored in a database comprising at least one of: a user database and an event database. 
     
     
         5 . The method of  claim 2 , wherein the backup copies are stored in a secondary storage device. 
     
     
         6 . The method of  claim 2 , wherein at least one word in the search query is categorized based at least on one of: its definition, part of speech, and location in the search query. 
     
     
         7 . The method of  claim 6 , further comprising determining whether the at least one word in the search query is one of: an event, a person's name, and a data type. 
     
     
         8 . The method of  claim 2 , wherein prior to performing the facial recognition operation, determining whether the search query is limited to searching only images comprising photographs and videos. 
     
     
         9 . The method of  claim 2 , further comprising performing natural language processing (NLP) on the search query. 
     
     
         10 . The method of  claim 2 , wherein the images to be backed up further comprises videos and photographs and storing a backup set of the videos and photographs. 
     
     
         11 . The method of  claim 10 , wherein each video from the backup set of videos is associated with a respective thumbnail image; and
 identifying at least one thumbnail image associated with a video from the backup set of videos; and   transmitting the video associated with the identified thumbnail image to the client computing device in response to the search query.   
     
     
         12 . The method of  claim 10 , wherein each video in the backup set of videos is associated with at least one image frame, and
 identifying at least one video from the backup videos by matching one or more facial features obtained from performing the facial recognition operations with the at least one image frame associated with the video; and   transmitting the at least one identified video to the client computing device in response to the search query.   
     
     
         13 . The method of  claim 2 , wherein the search query is categorized into at least one of: event indicator, source indicator, user indicator, and data type indicator. 
     
     
         14 . The method of  claim 2 , further comprising determining for each respective image from the plurality of images in the primary data, the respective location and the respective time, wherein the location comprises one of: global positioning system (GPS) coordinates, location value, and geographic location, and the respective time comprises a time at which the respective image was taken. 
     
     
         15 . A computer-implemented method for performing a targeted search on backup data, the method comprising:
 receiving a request to back up copies of primary data, wherein the primary data comprises a plurality of photographs wherein each photograph comprises metadata comprising a respective location and a respective time associated with each of the photographs, stored on one or more primary storage devices associated with a client computing device and wherein the client computing device is associated with a requesting user, and;   backing up the plurality of photographs from the primary data associated with the requesting user;   storing, as backup copies, user profiles and profile photographs associated with the user profiles, wherein the user profiles comprise at least the requesting user's profile and its associated profile photograph, and photographs from users other than the requesting user;   storing, as backup copies, the backed-up plurality of photographs from the primary data associated with the requesting user;   receiving, from the client computing device, a search query to search the stored backup copies;   identifying, based on the search query, a target user profile from the backup copies, wherein the target user profile is one of: the requesting user profile and the user profile other than the requesting user profile;   obtaining a target profile photograph associated with the target user profile, wherein the target profile photograph comprises one or more facial features;   performing facial recognition operations on the target profile photograph;   identifying at least one photograph among the backup copies of photographs that matches the one or more facial features from the target profile photograph; and   transmitting a search result comprising the identified at least one photograph to the client computing device.   
     
     
         16 . The method of  claim 15 , wherein the performing facial recognition operations comprises:
 detecting an image region in the target profile photograph that corresponds to the target user's face;   extracting one or more facial features from the detected image region in the target profile photograph.   
     
     
         17 . The method of  claim 15 , further comprising receiving the search request via a graphical user interface presented on the client computing device; and causing the search results to be displayed on the graphical user interface presented on the client computing device. 
     
     
         18 . The method of  claim 15 , wherein the request to back up the primary data made by one of: a storage policy and a user command. 
     
     
         19 . The method of  claim 15 , wherein the primary data further comprises a plurality of videos and storing the plurality of videos as backup copies. 
     
     
         20 . The method of  claim 19 , further comprising, identifying a set of videos from the backup copies of the plurality of videos, wherein each video is associated with a respective thumbnail image that matches one or more facial features obtain from performing the facial recognition operations; and further comprising including the set of identified videos in the search result transmitted to the client computing device. 
     
     
         21 . The method of  claim 19 , further comprising, identifying at least one video from the backup copies of the plurality of videos, wherein the identified at least one video matches one or more facial features obtain from performing the facial recognition operations; and further comprising including the identified at least one video in the search result transmitted to the client computing device.