Video content analysis for automatic demographics recognition of users and videos

Abstract

A video demographics analysis system selects a training set of videos to use to correlate viewer demographics and video content data. The video demographics analysis system extracts demographic data from viewer profiles related to videos in the training set and creates a set of demographic distributions, and also extracts video data from videos in the training set. The video demographics analysis system correlates the viewer demographics with the video data of videos viewed by that viewer. Using the prediction model produced by the machine learning process, a new video about which there is no a priori knowledge can be associated with a predicted demographic distribution specifying probabilities of the video appealing to different types of people within a given demographic category, such as people of different ages within an age demographic category.

1 . A computer-implemented method performed by a video hosting website, the method comprising:
 accessing a plurality of content items containing audiovisual content and received from client devices of a plurality of users over a computer network;   generating demographic distributions for at least one demographic attribute using viewer demographic data from user profiles of a plurality of users who have played the content items, the demographic distributions quantifying viewership of the content items for different values of the at least one demographic attribute;   generating feature vectors at least in part from audiovisual content of video frames of the content items, the feature vectors comprising a list of content attribute values;   generating, by a computer system, a prediction model that correlates the feature vectors and the demographic distributions;   determining that a first user lacks a demographic profile;   identifying a first plurality of content items played by the first user;   identifying, for the first plurality of content items, demographic attribute values associated with the first plurality of content items by applying the prediction model to a set of feature vectors generated at least in part from video frames of the first plurality of content items;   estimating demographic attributes values of the first user using the identified demographic attribute values of the first plurality of content items;   generating a list of recommended content items to the first user based on the estimated demographic attribute values of the first user; and   providing the generated list of recommended content items to the first user.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein the at least one demographic attribute comprises one of age and gender. 
     
     
         3 . The computer-implemented method of  claim 1 , wherein the at least one demographic attribute comprises one of occupation, household income, and location. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein the prediction model is generated using support vector machines. 
     
     
         5 . The computer-implemented method of  claim 1 , further comprising altering the generated feature vectors using a dimensionality reduction algorithm. 
     
     
         6 . The computer-implemented method of  claim 1 , wherein the generated feature vectors further include feature vectors generated using audio content of the content items. 
     
     
         7 . The computer-implemented method of  claim 1 , wherein the generated feature vectors include feature vectors that are generated using metadata associated with the content items. 
     
     
         8 . The computer-implemented method of  claim 1 , wherein the content items comprise videos, the method further comprising:
 performing object segmentation on the video frames of the content items, thereby identifying visual objects of the frames;   wherein generating feature vectors comprises generating feature vectors for the identified visual objects.   
     
     
         9 . A non-transitory computer readable storage medium storing a computer program executable by a processor of a video hosting website, the computer program comprising:
 instructions for accessing a plurality of videos and received from client devices of a plurality of users over a computer network;   instructions for creating demographic distributions for at least one demographic attribute using viewer demographic data from user profiles of a plurality of viewers of the videos, the demographic distributions quantifying viewership of the content items for different values of the at least one demographic attribute;   instructions for generating feature vectors from video content of video frames of the videos, the feature vectors comprising a list of content attribute values;   instructions for generating a prediction model that correlates the feature vectors and the demographic distributions;   instructions for determining that a first user lacks a demographic profile;   instructions for identifying a first plurality of content items played by the first user;   instructions for identifying, for the first plurality of content items, demographic attribute values associated with the first plurality of content items by applying the prediction model to a set of feature vectors generated at least in part from video frames of the first plurality of content items;   instructions for estimating demographic attributes values of the first user using the identified demographic attribute values of the first plurality of content items;   instructions for generating a list of recommended content items to the first user based on the estimated demographic attribute values of the first user; and   instructions for providing the generated list of recommended content items to the first user.   
     
     
         10 . The non-transitory computer readable storage medium of  claim 9 , wherein the at least one demographic attribute comprises one of age and gender. 
     
     
         11 . The non-transitory computer readable storage medium of  claim 9 , wherein the at least one demographic attribute comprises one of occupation, household income, and location. 
     
     
         12 . The non-transitory computer readable storage medium of  claim 9 , wherein the prediction model is generated using support vector machines. 
     
     
         13 . The non-transitory computer readable storage medium of  claim 9 , the computer program further comprising instructions for altering the generated feature vectors using a dimensionality reduction algorithm. 
     
     
         14 . The non-transitory computer readable storage medium of  claim 9 , wherein the generated feature vectors further include feature vectors generated using audio content of the videos. 
     
     
         15 . The non-transitory computer readable storage medium of  claim 9 , wherein the generated feature vectors include feature vectors that are generated using metadata associated with the videos. 
     
     
         16 . The non-transitory computer readable storage medium of  claim 9 , the computer program further comprising:
 instructions for performing object segmentation on the video frames of the videos, thereby identifying visual objects of the video frames;   wherein generating feature vectors based at least in part on the video content of video frames of the videos comprises generating feature vectors for the identified visual objects.   
     
     
         17 . A computer-implemented method performed by a video hosting website, the method comprising:
 accessing a prediction model that correlates viewer demographic attributes from user profiles of a plurality of viewers with feature vectors extracted from a plurality of content items played by a plurality of viewers;   determining that a first user lacks a demographic profile;   identifying a first plurality of content items played by the first user;   generating, from audiovisual content of video frames of the first plurality of content items, a set of feature vectors;   identifying, by a computer system, for the first plurality of content items, demographic attribute values associated with the first plurality of content items by applying the prediction model to the generated set of feature vectors;   estimating demographic attributes values of the first user using the identified demographic attribute values of the first plurality of content items;   generating a list of recommended content items to the first user based on the estimated demographic attribute values of the first user; and   providing the generated list of recommended content items to the first user.   
     
     
         18 . The computer-implemented method of  claim 17 , wherein identifying the demographic attribute values comprises:
 identifying a set of feature vectors from of the prediction model that is most similar to the generated set of feature vectors from; and   identifying, in the prediction model, demographic attribute values most strongly correlated with the identified feature vectors from.   
     
     
         19 . The computer-implemented method of  claim 17 , wherein the viewer demographic attributes comprise at least one of age and gender. 
     
     
         20 . The computer-implemented method of  claim 17 , wherein the viewer demographic attributes comprise at least one of occupation, household income, and location.