Training data quality for spam classification

Abstract

In one embodiment, a method includes accessing posts in a social-networking system. Each of the posts is unlabeled with respect to whether the post is known to be spam. The method also includes determining a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed. The method further includes determining a first vector representation of the posting user and a second vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user. The method still further includes comparing the vector representations and building a machine learning model for automatically detecting spam posts in the social-networking system using a subset of the plurality of posts as non-spam training data.

1 - 20 . (canceled) 
     
     
         21 . A method comprising:
 by a computing device, accessing a training data comprising a plurality of posts in a social-networking system that have been flagged by users of the social-networking system as being spam;   by the computing device, training a first machine-learning model using the training data;   by the computing device, filtering the training data to identify a first subset of the plurality of posts as spam posts and a second subset of the plurality of posts as non-spam posts; and   by the computing device, training a second machine-learning model using the filtered training data, wherein:
 the first machine-learning model is trained to predict whether a particular post is likely to be flagged by the users of the social-networking system as being spam; and 
 the second machine-learning model is trained to confirm whether the particular post is spam or not spam. 
   
     
     
         22 . The method of  claim 21 , further comprising:
 accessing a post on the social-networking system;   predicting, based on the first machine learning model, that the post is likely to be flagged by the users of the social-networking system as being spam; and   determining, based on the second machine learning model, that the post is spam.   
     
     
         23 . The method of  claim 21 , wherein filtering the training data comprises removing the second subset of posts that are identified as non-spam posts from the plurality of posts that have been flagged by the users of the social-networking system as being spam. 
     
     
         24 . The method of  claim 21 , wherein filtering the training data to identify the first subset of the plurality of posts as spam posts and the second subset of the plurality of posts as non-spam posts comprises:
 determining, for each of the plurality of posts, a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed;   determining, for each of the plurality of posts, a first feature vector representation of the posting user and a second feature vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user;   comparing, for each of the plurality of posts, the first feature vector representation of the posting user to the second feature vector representation of the recipient user;   identifying the first subset of the plurality of posts as spam posts based on first feature vector representations of posting users associated with the first subset of posts as sufficiently different from second feature vector representations of recipient users associated with the first subset of posts according to a predefined metric; and   identifying the second subset of the plurality of posts as non-spam posts based on first feature vector representations of posting users associated with the second subset of posts as sufficiently similar to second feature vector representations of recipient users associated with the second subset of posts according to the predefined metric.   
     
     
         25 . The method of  claim 24 , wherein the recipient user is associated with a concept node in a social graph within the social-networking system. 
     
     
         26 . The method of  claim 24 , wherein the predefined metric comprises a distance threshold between vectors in a multi-dimensional mapping space. 
     
     
         27 . The method of  claim 24 , wherein the predefined metric comprises a particular pattern of interactions defined by each first and second vector representation. 
     
     
         28 . The method of  claim 21 , wherein one or more of the plurality of posts is associated with negative user feedback. 
     
     
         29 . The method of  claim 28 , wherein the negative user feedback is an indication, received from one or more users of the social-networking system, that the one or more of the plurality of posts are unsolicited posts. 
     
     
         30 . The method of  claim 21 , wherein the second machine-learning model is further trained based on second training data comprising a plurality of labeled spam posts and a plurality of labeled non-spam posts. 
     
     
         31 . One or more computer-readable non-transitory storage media embodying software that is operable when executed to:
 access a training data comprising a plurality of posts in a social-networking system that have been flagged by users of the social-networking system as being spam;   train a first machine-learning model using the training data;   filter the training data to identify a first subset of the plurality of posts as spam posts and a second subset of the plurality of posts as non-spam posts; and   train a second machine-learning model using the filtered training data, wherein:
 the first machine-learning model is trained to predict whether a particular post is likely to be flagged by the users of the social-networking system as being spam; and 
 the second machine-learning model is trained to confirm whether the particular post is spam or not spam. 
   
     
     
         32 . The media of  claim 31 , wherein the software is further operable when executed to:
 access a post on the social-networking system;   predict, based on the first machine learning model, that the post is likely to be flagged by the users of the social-networking system as being spam; and   determine, based on the second machine learning model, that the post is spam.   
     
     
         33 . The media of  claim 31 , wherein filtering the training data comprises removing the second subset of posts that are identified as non-spam posts from the plurality of posts that have been flagged by the users of the social-networking system as being spam. 
     
     
         34 . The media of  claim 31 , wherein to filter the training data to identify the first subset of the plurality of posts as spam posts and the second subset of the plurality of posts as non-spam posts, the software is further operable when executed to:
 determine, for each of the plurality of posts, a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed;   determine, for each of the plurality of posts, a first feature vector representation of the posting user and a second feature vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user;   compare, for each of the plurality of posts, the first feature vector representation of the posting user to the second feature vector representation of the recipient user;   identify the first subset of the plurality of posts as spam posts based on first feature vector representations of posting users associated with the first subset of posts as sufficiently different from second feature vector representations of recipient users associated with the first subset of posts according to a predefined metric; and   identify the second subset of the plurality of posts as non-spam posts based on first feature vector representations of posting users associated with the second subset of posts as sufficiently similar to second feature vector representations of recipient users associated with the second subset of posts according to the predefined metric.   
     
     
         35 . The media of  claim 31 , wherein the second machine-learning model is further trained based on second training data comprising a plurality of labeled spam posts and a plurality of labeled non- spam posts. 
     
     
         36 . A system comprising:
 one or more processors; and   one or more computer-readable non-transitory storage media coupled to one or more of the processors and comprising instructions operable when executed by one or more of the processors to cause the system to:   access a training data comprising a plurality of posts in a social-networking system that have been flagged by users of the social-networking system as being spam;   train a first machine-learning model using the training data;   filter the training data to identify a first subset of the plurality of posts as spam posts and a second subset of the plurality of posts as non-spam posts; and   train a second machine-learning model using the filtered training data, wherein:
 the first machine-learning model is trained to predict whether a particular post is likely to be flagged by the users of the social-networking system as being spam; and 
 the second machine-learning model is trained to confirm whether the particular post is spam or not spam. 
   
     
     
         37 . The system of  claim 36 , wherein the processors are further operable when executing the instructions to:
 access a post on the social-networking system;   predict, based on the first machine learning model, that the post is likely to be flagged by the users of the social-networking system as being spam; and   determine, based on the second machine learning model, that the post is spam.   
     
     
         38 . The system of  claim 36 , wherein filtering the training data comprises removing the second subset of posts that are identified as non-spam posts from the plurality of posts that have been flagged by the users of the social-networking system as being spam. 
     
     
         39 . The system of  claim 36 , wherein to filter the training data to identify the first subset of the plurality of posts as spam posts and the second subset of the plurality of posts as non-spam posts, the processors are further operable when executing the instructions to:
 determine, for each of the plurality of posts, a posting user who submitted the post to the social-networking system and a recipient user to whom the post is addressed;   determine, for each of the plurality of posts, a first feature vector representation of the posting user and a second feature vector representation of the recipient user based on one or more features associated with the post, the posting user, and the recipient user;   compare, for each of the plurality of posts, the first feature vector representation of the posting user to the second feature vector representation of the recipient user;   identify the first subset of the plurality of posts as spam posts based on first feature vector representations of posting users associated with the first subset of posts as sufficiently different from second feature vector representations of recipient users associated with the first subset of posts according to a predefined metric; and   identify the second subset of the plurality of posts as non-spam posts based on first feature vector representations of posting users associated with the second subset of posts as sufficiently similar to second feature vector representations of recipient users associated with the second subset of posts according to the predefined metric.   
     
     
         40 . The system of  claim 36 , wherein the second machine-learning model is further trained based on second training data comprising a plurality of labeled spam posts and a plurality of labeled non-spam posts.