Systems and methods for using video metadata to associate advertisements therewith

Abstract

A system for using metadata from a video signal to associate advertisements therewith, comprising (i) a segmentation system to divide the video signal into video clips, (ii) a digitizing system for digitizing the video clips, (iii) a feature extraction system for extracting audio and video features from each video clip, associating each audio feature with respective video clips, associating each video feature with respective video clips, and saving the audio and video features into an associated metadata file, (iv) a web interface to the feature extraction system for receiving the video clips, and (v) a database, wherein video signals and associated metadata files are stored and indexed, wherein the associated metadata file is provided when a video player requests the corresponding video signal, enabling selection of a relevant advertisement for presentment in conjunction with respective video clips based on the associated audio and video features of the respective video clip.

What is claimed is: 
     
         1 . A method, comprising:
 receiving end user characteristics metadata from a video display device associated with an end user;   receiving a time-based metadata file providing time-coded information associated with a video program to enable selection of appropriate advertisements for presentment at specific time-code locations of the video program;   receiving in real-time, an actual time-code location of the video program as the video program is being viewed on the video display device by the end user;   selecting in real-time, an advertisement based on the end user characteristics metadata, the time-based metadata file, and the actual time-code location as the video program is being viewed;   providing the advertisement as the video program is being viewed, over the Internet to the video display device, for real-time presentment of the advertisement on the video display device at the actual time-code location of the video program, the real-time presentment of the advertisement comprising presentment of the advertisement simultaneous and in non-interference with the video program.   
     
     
         2 . The method of  claim 1 , where the time-based metadata file includes at least identified extracted audio features, identified extracted video features, and a context for each video segment of a plurality of video segments comprising the video program. 
     
     
         3 . The method of  claim 1 , where the time-based metadata file includes identified extracted audio features generated by extracting audio features from the video program, identifying extracted audio features from the video program using audio processing, each of the audio features including corresponding start and stop time codes. 
     
     
         4 . The method of  claim 1 , where the time-based metadata file includes identified extracted video features generated by extracting video features from the video program, identifying extracted video features from the video program using visual processing, each of the video features including corresponding start and stop time codes 
     
     
         5 . The method of  claim 1 , where the time-based metadata file includes a context for each video segment of a plurality of video segments comprising the video program, the context for each video segment of the plurality of video segments automatically determined from analysis of identified extracted audio and video features generated at least in part by extracting audio and video features from the video program. 
     
     
         6 . The method of  claim 1 , where the time-based metadata file includes identified extracted video features that represent one or more of actors, characters, animals, objects, geographic locations, background, setting, theme, events, and scenes. 
     
     
         7 . The method of  claim 1 , where the time-based metadata file includes identified extracted audio features that represent one or more of words, speeches, dialogue, music, discrete sounds, and background noise. 
     
     
         8 . The method of  claim 1 , where the advertisement correlates to at least one of: (a) extracted video features obtained from the time-based metadata file that appears in the video program during the respective time code location or (b) extracted audio features obtained from the time-based metadata file that occurs in the video program during the respective time code location. 
     
     
         9 . The method of  claim 1 , where the time-based metadata file comprises one or more of: (a) identification of the video program; (b) a file name; (c) a digital signature; (d) the time-coded duration of the video program; (e) a keyword list; (f) a time-coded transcript; and (g) one or more segments of the video program, each segment having a corresponding start and stop time and each segment identifying video features and audio features associated with each said segment. 
     
     
         10 . The method of  claim 1 , wherein the advertisement comprises background information, detailed information about the video program, detailed information about actors, scenes, events, or locations that appear in the video program, or related video programs. 
     
     
         11 . One or more non-transitory machine-readable media storing instructions which, when executed by one or more processors, cause the one or more processors to perform:
 receiving end user characteristics metadata from a video display device associated with an end user;   receiving a time-based metadata file providing time-coded information associated with a video program to enable selection of appropriate advertisements for presentment at specific time-code locations of the video program;   receiving in real-time, an actual time-code location of the video program as the video program is being viewed on the video display device by the end user;   selecting in real-time, an advertisement based on the end user characteristics metadata, the time-based metadata file, and the actual time-code location as the video program is being viewed;   providing the advertisement as the video program is being viewed, over the Internet to the video display device, for real-time presentment of the advertisement on the video display device at the actual time-code location of the video program, the real-time presentment of the advertisement comprising presentment of the advertisement simultaneous and in non-interference with the video program.   
     
     
         12 . The media of  claim 11 , where the time-based metadata file includes at least identified extracted audio features, identified extracted video features, and a context for each video segment of a plurality of video segments comprising the video program. 
     
     
         13 . The media of  claim 11 , where the time-based metadata file includes identified extracted audio features generated by extracting audio features from the video program, identifying extracted audio features from the video program using audio processing, each of the audio features including corresponding start and stop time codes. 
     
     
         14 . The media of  claim 11 , where the time-based metadata file includes identified extracted video features generated by extracting video features from the video program, identifying extracted video features from the video program using visual processing, each of the video features including corresponding start and stop time codes 
     
     
         15 . The media of  claim 11 , where the time-based metadata file includes a context for each video segment of a plurality of video segments comprising the video program, the context for each video segment of the plurality of video segments automatically determined from analysis of identified extracted audio and video features generated at least in part by extracting audio and video features from the video program. 
     
     
         16 . The media of  claim 11 , where the time-based metadata file includes identified extracted video features that represent one or more of actors, characters, animals, objects, geographic locations, background, setting, theme, events, and scenes. 
     
     
         17 . The media of  claim 11 , where the time-based metadata file includes identified extracted audio features that represent one or more of words, speeches, dialogue, music, discrete sounds, and background noise. 
     
     
         18 . The media of  claim 11 , where the advertisement correlates to at least one of: (a) extracted video features obtained from the time-based metadata file that appears in the video program during the respective time code location or (b) extracted audio features obtained from the time-based metadata file that occurs in the video program during the respective time code location. 
     
     
         19 . The media of  claim 11 , where the time-based metadata file comprises one or more of: (a) identification of the video program; (b) a file name; (c) a digital signature; (d) the time-coded duration of the video program; (e) a keyword list; (f) a time-coded transcript; and (g) one or more segments of the video program, each segment having a corresponding start and stop time and each segment identifying video features and audio features associated with each said segment. 
     
     
         20 . The media of  claim 11 , wherein the advertisement comprises background information, detailed information about the video program, detailed information about actors, scenes, events, or locations that appear in the video program, or related video programs. 
     
     
         21 . An apparatus, comprising:
 a subsystem, implemented at least in part in hardware, that receives end user characteristics metadata from a video display device associated with an end user;   a subsystem, implemented at least in part in hardware, that receives a time-based metadata file providing time-coded information associated with a video program to enable selection of appropriate advertisements for presentment at specific time-code locations of the video program;   a subsystem, implemented at least in part in hardware, that receives in real-time, an actual time-code location of the video program as the video program is being viewed on the video display device by the end user;   a subsystem, implemented at least in part in hardware, that selects in real-time, an advertisement based on the end user characteristics metadata, the time-based metadata file, and the actual time-code location as the video program is being viewed;   a subsystem, implemented at least in part in hardware, that provides the advertisement as the video program is being viewed, over the Internet to the video display device, for real-time presentment of the advertisement on the video display device at the actual time-code location of the video program, the real-time presentment of the advertisement comprising presentment of the advertisement simultaneous and in non-interference with the video program.   
     
     
         22 . The apparatus of  claim 21 , where the time-based metadata file includes at least identified extracted audio features, identified extracted video features, and a context for each video segment of a plurality of video segments comprising the video program. 
     
     
         23 . The apparatus of  claim 21 , where the time-based metadata file includes identified extracted audio features generated by extracting audio features from the video program, identifying extracted audio features from the video program using audio processing, each of the audio features including corresponding start and stop time codes. 
     
     
         24 . The apparatus of  claim 21 , where the time-based metadata file includes identified extracted video features generated by extracting video features from the video program, identifying extracted video features from the video program using visual processing, each of the video features including corresponding start and stop time codes 
     
     
         25 . The apparatus of  claim 21 , where the time-based metadata file includes a context for each video segment of a plurality of video segments comprising the video program, the context for each video segment of the plurality of video segments automatically determined from analysis of identified extracted audio and video features generated at least in part by extracting audio and video features from the video program. 
     
     
         26 . The apparatus of  claim 21 , where the time-based metadata file includes identified extracted video features that represent one or more of actors, characters, animals, objects, geographic locations, background, setting, theme, events, and scenes. 
     
     
         27 . The apparatus of  claim 21 , where the time-based metadata file includes identified extracted audio features that represent one or more of words, speeches, dialogue, music, discrete sounds, and background noise. 
     
     
         28 . The apparatus of  claim 21 , where the advertisement correlates to at least one of: (a) extracted video features obtained from the time-based metadata file that appears in the video program during the respective time code location or (b) extracted audio features obtained from the time-based metadata file that occurs in the video program during the respective time code location. 
     
     
         29 . The apparatus of  claim 21 , where the time-based metadata file comprises one or more of: (a) identification of the video program; (b) a file name; (c) a digital signature; (d) the time-coded duration of the video program; (e) a keyword list; (f) a time-coded transcript; and (g) one or more segments of the video program, each segment having a corresponding start and stop time and each segment identifying video features and audio features associated with each said segment. 
     
     
         30 . The apparatus of  claim 21 , wherein the advertisement comprises background information, detailed information about the video program, detailed information about actors, scenes, events, or locations that appear in the video program, or related video programs.