Systems and methods for captioning content

Abstract

Systems, methods, and non-transitory computer-readable media can determine one or more chunks for a content item to be captioned. Each chunk can include one or more terms that describe at least a portion of the subject matter captured in the content item. One or more sentiments are determined based on the subject matter captured in the content item. One or more emotions are determined for the content item. At least one emoted caption is generated for the content item based at least in part on the one or more chunks, sentiments, and emotions. The emoted caption can include at least one term that conveys an emotion represented by the subject matter captured in the content item.

What is claimed is: 
     
         1 . A computer-implemented method comprising:
 determining, by a computing system, one or more chunks for a content item to be captioned, each chunk including one or more terms that describe at least a portion of the subject matter captured in the content item;   determining, by the computing system, one or more sentiments based on the subject matter captured in the content item;   determining, by the computing system, one or more emotions for the content item; and   generating, by the computing system, at least one emoted caption for the content item based at least in part on the one or more chunks, sentiments, and emotions, the emoted caption including at least one term that conveys an emotion represented by the subject matter captured in the content item.   
     
     
         2 . The computer-implemented method of  claim 1 , wherein determining one or more emotions for the content item further comprises:
 obtaining, by the computing system, a set of candidate emotions; and   determining, by the computing system, respective probabilities for each emotion in the set of candidate emotions based at least in part on the one or more chunks and the one or more sentiments, wherein a probability for an emotion measures a likelihood of the emotion appearing with a given chunk in the context of a given sentiment.   
     
     
         3 . The computer-implemented method of  claim 2 , wherein the probability associated with an emotion with respect to a given chunk and a given sentiment is based at least in part on a number of occurrences of the emotion appearing with the given chunk in the context of the given sentiment within a corpus of web documents. 
     
     
         4 . The computer-implemented method of  claim 1 , wherein generating the at least one emoted caption for the content item further comprises:
 generating, by the computing system, a respective score for each combination of the one or more chunks, the one or more sentiments, and the one or more emotions; and   identifying, by the computing system, one or more best scoring combinations to be included in the emoted caption for the content item.   
     
     
         5 . The computer-implemented method of  claim 4 , wherein generating the score for a combination of a given chunk, sentiment, and emotion further comprises:
 obtaining, by the computing system, a first probability associated with the chunk, the first probability measuring a likelihood of the chunk describing at least a portion of the subject matter captured in the content item;   obtaining, by the computing system, a second probability associated with the sentiment, the second probability measuring a likelihood of the sentiment describing at least a portion of the subject matter captured in the content item;   obtaining, by the computing system, a third probability associated with the emotion, the third probability measuring a likelihood of the emotion corresponding to the given chunk given the sentiment; and   multiplying, by the computing system, the first probability, the second probability, and the third probability to produce the score for the combination.   
     
     
         6 . The computer-implemented method of  claim 4 , wherein a score for a combination measures a likelihood with which the combination conveys an emotion represented by the subject matter captured in the content item. 
     
     
         7 . The computer-implemented method of  claim 1 , wherein determining one or more chunks for the content item to be captioned further comprises:
 training, by the computing system, a caption model to output chunks for the content item, the caption model being trained using a set of training content items that have each been labeled with one or more chunks that describe the subject matter captured in the training content item, wherein each chunk outputted by the caption model is associated with a respective probability.   
     
     
         8 . The computer-implemented method of  claim 1 , wherein determining one or more sentiments further comprises:
 training, by the computing system, a sentiment model to output sentiments for the content item, the sentiment model being trained using a set of training content items that have each been labeled with one or more sentiments that describe the subject matter captured in the training content item, wherein each sentiment outputted by the sentiment model is associated with a respective probability.   
     
     
         9 . The computer-implemented method of  claim 8 , wherein the sentiment model is trained to recognize a happy sentiment, a sad sentiment, an angry sentiment, a disgust sentiment, a surprise sentiment, and a fear sentiment. 
     
     
         10 . The computer-implemented method of  claim 8 , wherein the set of training content items were provided by users of a social networking system, each training content item having been labeled by a user that provided the training content item using one or more descriptive hash tags. 
     
     
         11 . A system comprising:
 at least one processor; and   a memory storing instructions that, when executed by the at least one processor, cause the system to perform:
 determining one or more chunks for a content item to be captioned, each chunk including one or more terms that describe at least a portion of the subject matter captured in the content item; 
 determining one or more sentiments based on the subject matter captured in the content item; 
 determining one or more emotions for the content item; and 
 generating at least one emoted caption for the content item based at least in part on the one or more chunks, sentiments, and emotions, the emoted caption including at least one term that conveys an emotion represented by the subject matter captured in the content item. 
   
     
     
         12 . The system of  claim 11 , wherein determining one or more emotions for the content item further causes the system to perform:
 obtaining a set of candidate emotions; and   determining respective probabilities for each emotion in the set of candidate emotions based at least in part on the one or more chunks and the one or more sentiments, wherein a probability for an emotion measures a likelihood of the emotion appearing with a given chunk in the context of a given sentiment.   
     
     
         13 . The system of  claim 12 , wherein the probability associated with an emotion with respect to a given chunk and a given sentiment is based at least in part on a number of occurrences of the emotion appearing with the given chunk in the context of the given sentiment within a corpus of web documents. 
     
     
         14 . The system of  claim 11 , wherein generating the at least one emoted caption for the content item further causes the system to perform:
 generating a respective score for each combination of the one or more chunks, the one or more sentiments, and the one or more emotions; and   identifying one or more best scoring combinations to be included in the emoted caption for the content item.   
     
     
         15 . The system of  claim 14 , wherein generating the score for a combination of a given chunk, sentiment, and emotion further causes the system to perform:
 obtaining a first probability associated with the chunk, the first probability measuring a likelihood of the chunk describing at least a portion of the subject matter captured in the content item;   obtaining a second probability associated with the sentiment, the second probability measuring a likelihood of the sentiment describing at least a portion of the subject matter captured in the content item;   obtaining a third probability associated with the emotion, the third probability measuring a likelihood of the emotion corresponding to the given chunk given the sentiment; and   multiplying the first probability, the second probability, and the third probability to produce the score for the combination.   
     
     
         16 . A non-transitory computer-readable storage medium including instructions that, when executed by at least one processor of a computing system, cause the computing system to perform a method comprising:
 determining one or more chunks for a content item to be captioned, each chunk including one or more terms that describe at least a portion of the subject matter captured in the content item;   determining one or more sentiments based on the subject matter captured in the content item;   determining one or more emotions for the content item; and   generating at least one emoted caption for the content item based at least in part on the one or more chunks, sentiments, and emotions, the emoted caption including at least one term that conveys an emotion represented by the subject matter captured in the content item.   
     
     
         17 . The non-transitory computer-readable storage medium of  claim 16 , wherein determining one or more emotions for the content item further causes the computing system to perform:
 obtaining a set of candidate emotions; and   determining respective probabilities for each emotion in the set of candidate emotions based at least in part on the one or more chunks and the one or more sentiments, wherein a probability for an emotion measures a likelihood of the emotion appearing with a given chunk in the context of a given sentiment.   
     
     
         18 . The non-transitory computer-readable storage medium of  claim 17 , wherein the probability associated with an emotion with respect to a given chunk and a given sentiment is based at least in part on a number of occurrences of the emotion appearing with the given chunk in the context of the given sentiment within a corpus of web documents. 
     
     
         19 . The non-transitory computer-readable storage medium of  claim 16 , wherein generating the at least one emoted caption for the content item further causes the computing system to perform:
 generating a respective score for each combination of the one or more chunks, the one or more sentiments, and the one or more emotions; and   identifying one or more best scoring combinations to be included in the emoted caption for the content item.   
     
     
         20 . The non-transitory computer-readable storage medium of  claim 19 , wherein generating the score for a combination of a given chunk, sentiment, and emotion further causes the computing system to perform:
 obtaining a first probability associated with the chunk, the first probability measuring a likelihood of the chunk describing at least a portion of the subject matter captured in the content item;   obtaining a second probability associated with the sentiment, the second probability measuring a likelihood of the sentiment describing at least a portion of the subject matter captured in the content item;   obtaining a third probability associated with the emotion, the third probability measuring a likelihood of the emotion corresponding to the given chunk given the sentiment; and   multiplying the first probability, the second probability, and the third probability to produce the score for the combination.