Detection of cast members in video content

Abstract

Disclosed are various embodiments for detection of cast members in video content such as movies, television shows, and other programs. Data indicating cast members who appear in a video program is obtained. Each cast member is associated with a reference image depicting a face of the cast member. A frame is obtained from the video program, and a face is detected in the frame. The frame can correspond to a scene in the video program. The detected face in the frame is recognized as being a particular cast member based at least in part on the reference image depicting the cast member. An association between the cast member and the frame is generated in response to the detected face in the frame being recognized as the cast member.

Therefore, the following is claimed: 
     
         1 . A non-transitory computer-readable medium embodying a program that, when executed by at least one computing device, causes the at least one computing device to at least:
 sample a first frame from a scene in a video program;   detect a face in the first frame of the scene;   recognize the face in the first frame as being one of a plurality of cast members known to appear in the video program based at least in part on a plurality of reference images corresponding to the plurality of cast members;   generate a first association between the one of the plurality of cast members and the first frame when the face is recognized as being the one of the plurality of cast members; and   generate a second association between the one of the plurality of cast members and a second frame from the scene in the video program based at least in part on the first association, a third frame of the video program, and a temporal smoothing factor.   
     
     
         2 . The non-transitory computer-readable medium of  claim 1 , wherein the program further causes the at least one computing device to at least generate a third association between the one of the plurality of cast members and the scene based at least in part on the first association and the second association. 
     
     
         3 . The non-transitory computer-readable medium of  claim 1 , wherein the program further causes the at least one computing device to at least generate a third association between the one of the plurality of cast members and a fourth frame from the scene in the video program, the third association indicating the one of the plurality of cast members is predicted to appear in the fourth frame. 
     
     
         4 . The non-transitory computer-readable medium of  claim 1 , wherein the program further causes the at least one computing device to at least detect an ending frame of the scene based at least in part on a scene break. 
     
     
         5 . The non-transitory computer-readable medium of  claim 4 , wherein the program further causes the at least one computing device to at least identify the scene break by detecting a change in contrast between at least two frames of the video program. 
     
     
         6 . The non-transitory computer-readable medium of  claim 1 , wherein the scene corresponds to a plurality of consecutive frames that comprises the first frame and the second frame. 
     
     
         7 . The non-transitory computer-readable medium of  claim 1 , wherein the scene corresponds to a distinct plot element of the video program. 
     
     
         8 . A system, comprising:
 a data store; and   at least one computing device comprising a hardware processor, the at least one computing device being in communication with the data store, the at least one computing device being configured to at least:
 receive data indicating a plurality of persons that are known to appear in a video program, an individual person of the plurality of persons being associated with a reference image depicting a respective face of the individual person; 
 identify a frame from the video program; 
 detect a face in the frame; 
 recognize the face in the frame as being the individual person based at least in part on the reference image depicting the respective face of the individual person; and 
 generate an association between the individual person and the frame in response to the face in the frame being recognized as being the individual person. 
   
     
     
         9 . The system of  claim 8 , wherein the at least one computing device is further configured to at least:
 submit the face in the frame for a manual review; and   in response to the manual review indicating that the face matches the reference image, update the data indicating the plurality of persons to improve accuracy of another recognition in another frame.   
     
     
         10 . The system of  claim 9 , wherein the other frame corresponds to a second video program. 
     
     
         11 . The system of  claim 8 , wherein the at least one computing device is further configured to at least:
 detect a second face in a second frame;   recognize the second face in the second frame as being the individual person based at least in part on the reference image depicting the respective face of the individual person; and   generate a second association between the individual person and the second frame.   
     
     
         12 . The system of  claim 11 , wherein the at least one computing device is further configured to at least generate a third association between the individual person and a third frame based at least in part on a temporal smoothing factor, wherein the individual person is unrecognized in the third frame, and the third frame is located chronologically between the frame and the second frame. 
     
     
         13 . The system of  claim 12 , wherein a count of a number of frames between the frame and the second frame is at or below a threshold count of video frames in which a person may be unrecognized and be associated with a particular frame. 
     
     
         14 . A method, comprising:
 obtaining, by at least one computing device, a frame from a scene of a video program;   identifying, by the at least one computing device, a face in the frame;   recognizing, by the at least one computing device, the face in the frame as one of a plurality of persons based at least in part on a reference image depicting a person; and   generating, by the at least one computing device, an association between the person and the scene in response to the face in the frame being recognized as being the person.   
     
     
         15 . The method of  claim 14 , further comprising:
 obtaining, by the at least one computing device, a second frame from video program;   detecting, by the at least one computing device, a second face in the second frame;   recognizing, by the at least one computing device, the second face in the frame as the one of the plurality of persons; and   determining, by the at least one computing device, a trajectory of the one of the plurality of persons in the video program based at least in part on the frame and the second frame.   
     
     
         16 . The method of  claim 15 , further comprising:
 obtaining, by the at least one computing device, a third frame from video program; and   detecting, by the at least one computing device, a third face in the third frame based at least in part on the trajectory of the one of the plurality of persons.   
     
     
         17 . The method of  claim 14 , further comprising:
 obtaining, by the at least one computing device, a subset of a plurality of frames of the video program by selecting a single respective frame of the plurality of frames for an individual predefined time interval;   identifying, by the at least one computing device, a respective face in individual ones of the subset of the plurality of frames; and   recognizing, by the at least one computing device, the respective face in the individual ones of the subset of the plurality of frames as a respective one of the plurality of persons based at least in part on a plurality of reference images depicting a particular person known to appear in the video program.   
     
     
         18 . The method of  claim 14 , further comprising:
 Identifying, by the at least one computing device, a scene break in the scene by detecting a change in contrast between at least two frames of the scene; and   detecting, by the at least one computing device, an ending frame of the scene based at least in part on the scene break.   
     
     
         19 . The method of  claim 14 , further comprising:
 providing, by the at least one computing device, the face in the frame for a manual review;   receiving, by the at least one computing device, an indication that the face matches the reference image based at least in part on the manual review;   updating, by the at least one computing device, data comprising the reference image depicting the person based at least in part on the frame; and   recognizing, by the at least one computing device, another face in another video program based at least in part on the updated data.   
     
     
         20 . The method of  claim 14 , wherein recognizing the face in the frame as the one of the plurality of persons further comprises:
 identifying, by the at least one computing device, a portion of the frame based at least in part on the face identified in the frame; and   performing, by the at least one computing device, a facial recognition on the portion of the frame.