Spam flood detection methodologies

Abstract

A computer-implemented method and system are provided in which characteristics of a website are analyzed to determine whether the website represents a potential source of spam, and, in response to the analyzing, flags content of the website as spam content. To analyze the website, a total number of posts associated with the website is computed and a publication frequency for the total number of posts can then be calculated. Based on the computed total number and the calculated publication frequency, it can be determined whether the website in its entirety represents spam content. For instance, the calculated publication frequency can be compared to a threshold frequency, and when the calculated publication frequency is greater than the threshold frequency, the website in its entirety as can be identified and flagged as spam content.

What is claimed is: 
     
       1. A computer-implemented method for identifying a website as spam content, the method comprising:
 analyzing characteristics of the website to determine whether the website represents a potential source of spam content, wherein the analyzing comprises:
 computing a total number of posts that appear within the website; 
 calculating a post publication frequency that indicates a ratio of the total number of posts to a publication time period; and 
 determining whether the web site in its entirety represents spam content, based on the computed total number and the calculated post publication frequency, wherein the determining comprises:
 comparing the computed total number of posts to a threshold number; 
 determining whether the computed total number of posts is greater than or equal to a threshold number; 
 comparing the calculated post publication frequency to a threshold frequency when the computed total number of posts is greater than or equal to the threshold number; and 
 identifying the website in its entirety as spam content when the computed total number of posts is greater than or equal to the threshold number and the calculated post publication frequency is greater than the threshold frequency; and 
 
 
 flagging the website in its entirety as spam content. 
 
     
     
       2. The computer-implemented method of  claim 1 , wherein the determining further comprises:
 when the computed total number is less than the threshold number, identifying the website in its entirety as non-spam content. 
 
     
     
       3. The computer-implemented method of  claim 1 , wherein the threshold frequency is less than five posts per minute, and wherein the threshold number is within a range of about 500 to 1000. 
     
     
       4. The computer-implemented method of  claim 1 , wherein the determining comprises:
 when the calculated post publication frequency is less than or equal to the threshold frequency, identifying the website in its entirety as non-spam content. 
 
     
     
       5. The computer-implemented method of  claim 1 , wherein calculating the post publication frequency comprises:
 calculating the post publication frequency in posts per minute based on a ratio of the total number of posts to a difference between a publication time of a newest post in the website and a publication time of an oldest post in the website. 
 
     
     
       6. The computer-implemented method of  claim 1 , further comprising:
 filtering the spam content from stored webcrawler data to obtain filtered webcrawler data; and 
 outputting the filtered webcrawler data for presentation to a user. 
 
     
     
       7. A computing system comprising a processor and a memory having computer-executable instructions stored thereon that, when executed by the processor, cause the computing system to:
 analyze characteristics of a website to determine whether the website represents a potential source of spam content by causing the computing system to: compute a total number of posts that appear within the website; calculate a post publication frequency that indicates a ratio of the total number of posts to a publication time period; and determine whether the website in its entirety represents spam content, based on the computed total number and the calculated post publication frequency by: compare the computed total number of posts to a threshold number; determine whether the computed total number of posts is greater than or equal to a threshold number; compare the calculated post publication frequency to a threshold frequency when the computed total number of posts is greater than or equal to the threshold number; and identify the website in its entirety as spam content when the computed total number of posts is greater than or equal to the threshold number and the calculated post publication frequency is greater than the threshold frequency; and 
 flag the website in its entirety as spam content. 
 
     
     
       8. The computing system of  claim 7  wherein the computer-executable instructions, when executed by the processor, cause the computing system to:
 identify the website in its entirety as non-spam content when the computed total number is less than the threshold number. 
 
     
     
       9. The computing system of  claim 7  wherein the computer-executable instructions, when executed by the processor, cause the computing system to:
 when the calculated post publication frequency is less than or equal to the threshold frequency, identify the website in its entirety as non-spam content. 
 
     
     
       10. The computing system of  claim 9 , wherein the threshold frequency is less than five posts per minute, and wherein the threshold number is within a range of about 500 to 1000. 
     
     
       11. The computing system of  claim 7  wherein the computer-executable instructions, when executed by the processor, cause the computing system to:
 calculate the post publication frequency by calculating the post publication frequency in posts per minute based on a ratio of the total number of posts to a difference between a publication time of a newest post in the website and a publication time of an oldest post in the web site. 
 
     
     
       12. The computing system of  claim 9  wherein the computer-executable instructions, when executed by the processor, cause the computing system to:
 filter the spam content from stored webcrawler data to obtain filtered webcrawler data; and 
 output the filtered webcrawler data for presentation to a user. 
 
     
     
       13. A tangible and non-transitory computer readable medium having computer-executable instructions stored thereon that, when executed by a processor, perform a method comprising:
 analyzing characteristics of a website to determine whether the website represents a potential source of spam content by:
 computing a total number of posts that appear within the website; 
 calculating a post publication frequency that indicates a ratio of the total number of posts to a publication time period; and 
 determining whether the website in its entirety represents spam content, based on the computed total number of posts and the calculated post publication frequency, wherein the determining comprises:
 comparing the computed total number of posts to a threshold number; 
 
 determining whether the computed total number of posts is greater than or equal to a threshold number; comparing the calculated post publication frequency to a threshold frequency when the computed total number of posts is greater than or equal to the threshold number; and identifying the website in its entirety as spam content when the computed total number of posts is greater than or equal to the threshold number and the calculated post publication frequency is greater than the threshold frequency; and 
 
 flagging the website in its entirety as spam content. 
 
     
     
       14. The tangible and non-transitory computer readable medium of  claim 13 , wherein the determining further comprises:
 when the computed total number is less than the threshold number, identifying the website in its entirety as non-spam content. 
 
     
     
       15. The tangible and non-transitory computer readable medium of  claim 13 , wherein the threshold frequency is less than five posts per minute, and wherein the threshold number is within a range of about 500 to 1000. 
     
     
       16. The tangible and non-transitory computer readable medium of  claim 13 , wherein the determining comprises:
 when the calculated post publication frequency is less than or equal to the threshold frequency, identifying the website in its entirety as non-spam content. 
 
     
     
       17. The tangible and non-transitory computer readable medium of  claim 13 , wherein calculating the post publication frequency comprises:
 calculating the post publication frequency in posts per minute based on a ratio of the total number of posts to a difference between a publication time of a newest post in the website and a publication time of an oldest post in the website. 
 
     
     
       18. The tangible and non-transitory computer readable medium of  claim 13 , further comprising:
 filtering the spam content from stored webcrawler data to obtain filtered webcrawler data; and 
 outputting the filtered webcrawler data for presentation to a user. 
 
     
     
       19. The computer-implemented method of  claim 1 , where the website is a blog, and wherein each post is a discrete message published within the blog.