Enriching online videos by content detection, searching, and information aggregation

Abstract

Many internet users consume content through online videos. For example, users may view movies, television shows, music videos, and/or homemade videos. It may be advantageous to provide additional information to users consuming the online videos. Unfortunately, many current techniques may be unable to provide additional information relevant to the online videos from outside sources. Accordingly, one or more systems and/or techniques for determining a set of additional information relevant to an online video are disclosed herein. In particular, visual, textual, audio, and/or other features may be extracted from an online video (e.g., original content of the online video and/or embedded advertisements). Using the extracted features, additional information (e.g., images, advertisements, etc.) may be determined based upon matching the extracted features with content of a database. The additional information may be presented to a user consuming the online video.

1 .- 20 . (canceled) 
     
     
         21 . A method comprising:
 extracting, by one or more processors, a set of features from promotional content generated by a first content provider and associated with an online video, the set of features including at least one of a textual feature, a visual feature, or an audio feature;   searching, by the one or more processors and based at least in part on the set of features, a database to identify additional information stored in the database, the additional information being generated by a second content provider different than the first content provider;   identifying, by the one or more processors, a candidate from the additional information; and   outputting, via communication connections, the candidate.   
     
     
         22 . The method of  claim 21 , wherein the candidate comprises at least one of a textual feature, a visual feature, or an audio feature. 
     
     
         23 . The method of  claim 21 , further comprising:
 identifying an additional candidate from the additional information;   aggregating the candidate and the additional candidate into an aggregate video; and   presenting the aggregate video and the promotional content sequentially or simultaneously.   
     
     
         24 . The method of  claim 21 , the visual feature comprising at least one of a texture, a color histogram, or a SIFT descriptor. 
     
     
         25 . The method of  claim 21 , the extracting the feature comprising at least one of:
 extracting the textual feature using an OCR text recognition technique; or   extracting the visual feature using a scale-invariant feature transform.   
     
     
         26 . The method of  claim 21 , wherein the outputting the candidate comprises at least one of:
 presenting, by a display associated with the one or more processors, the candidate as a video;   presenting, by the display, the candidate as an animation;   presenting, by the display, the candidate as an image; or   presenting, by the display, the candidate as text.   
     
     
         27 . The method of  claim 21 , the extracting the set of features comprising:
 parsing the promotional content associated with the online video into one or more portions, each of the one or more portions comprising at least one frame;   extracting a keyframe from at least one of the one or more portions; and   extracting the set of features from the keyframe.   
     
     
         28 . The method of  claim 21 , the extracting the set of features comprising:
 parsing the promotional content into one or more portions, each of the one or more portions including a set of frames; and
 selecting a middle frame as a keyframe from at least one of the set of frames of the one or more portions; or 
 selecting a frame having a video quality above a threshold quality as the keyframe. 
   
     
     
         29 . A system comprising:
 one or more processors;   memory communicatively coupled to the one or more processors, the memory storing instructions that, when executed by the one or more processors, cause the system to perform operations comprising:
 extracting a set of features from promotional content generated by a first content provider and associated with an online video, the set of features including at least one of a textual feature, a visual feature, or an audio feature; 
 searching, based at least in part on the set of features, a database to identify additional information stored in the database, the additional information being generated by a second content provider different than the first content provider; and 
 identifying a candidate from the additional information; and 
 outputting the candidate. 
   
     
     
         30 . The system of  claim 29 , wherein outputting the candidate comprises presenting the candidate and the promotional content sequentially or simultaneously. 
     
     
         31 . The system of  claim 29 , the operations further comprising selecting a frame of the promotional content based at least in part on the frame having a video quality above a threshold video quality. 
     
     
         32 . The system of  claim 29 , wherein searching the database to identify the additional information comprises executing a multi-modal relevance matching algorithm to determine the additional information based at least in part on the feature. 
     
     
         33 . The system of  claim 32 , wherein executing the multi-model relevance matching algorithm includes:
 executing a text-based search algorithm upon the database using the textual feature to determine a first list of additional information candidates;   executing a visual feature matching algorithm upon the database using the visual feature to determine a second list of additional information candidates; and   performing a linear combination of the first list and the second list to generate the additional information.   
     
     
         34 . One or more computer-readable storage media storing instructions that, when executed by one or more processors, perform acts comprising:
 extracting a set of features from promotional content generated by a first content provider and associated with an online video, the set of features including at least one of a textual feature, a visual feature, or an audio feature;   searching, based on the set of features, a database to identify additional information stored in the database, the additional information being generated by a second content provider different than the first content provider;   identifying a candidate from the additional information; and   outputting the candidate.   
     
     
         35 . The one or more computer-readable storage media of  claim 34 , the acts further comprising:
 parsing the promotional content into one or more portions, each of the one or more portions including a set of frames; and   extracting a keyframe from at least one of the set of frames of the one or more portions.   
     
     
         36 . The one or more computer-readable storage media of  claim 35 , wherein extracting the keyframe comprises:
 selecting a middle frame as a keyframe from the at least one of the set of frames of the one or more portions; or   selecting a frame from the at least one of the set of frames having a video quality above a threshold quality as the keyframe.   
     
     
         37 . The one or more computer-readable storage media of  claim 34 , the acts further comprising:
 identifying an additional candidate from the additional information;   aggregating the candidate and the additional candidate into an aggregate video; and   presenting the aggregate video and the promotional content sequentially or simultaneously.   
     
     
         38 . The one or more computer-readable storage media of  claim 34 , wherein outputting the candidate comprises outputting the candidate and the promotional content sequentially or simultaneously. 
     
     
         39 . The one or more computer-readable storage media of  claim 34 , wherein searching the database to identify the additional information comprises executing a multi-modal relevance matching algorithm to determine the additional information based at least in part on the feature. 
     
     
         40 . The one or more computer-readable storage media of  claim 39 , wherein executing the multi-model relevance matching algorithm includes:
 executing a text-based search algorithm upon the database using the textual feature to determine a first list of additional information candidates;   executing a visual feature matching algorithm upon the database using the visual feature to determine a second list of additional information candidates; and   performing a linear combination of the first list and the second list to generate the additional information.