Real-time event detection on social media streams

Abstract

According to an aspect, a method for event detection on social data streams includes receiving a stream of messages exchanged on a messaging platform, and detecting an event from the stream of messages, which may include detecting a first cluster group of trending entities over a first period of time, detecting a second group of trending entities over a second period of time, and generating a cluster chain by linking the second cluster group with the first cluster group, where the cluster chain represents the detected event over the first and second periods of time. The method includes storing the event as the cluster chain in a memory device on the messaging platform.

1 . A messaging system comprising:
 a messaging platform configured to exchange, over a network, messages to computing devices, the messaging platform including an event detector configured to execute in a first operational mode and a second operational mode,   wherein the event detector, in the first operational, is configured to:
 execute an event detection algorithm on at least a portion of an evaluation dataset stream to generate at least one first cluster chain for varying values of a control parameter; and 
 compute a performance metric regarding the execution of the event detection algorithm for the varying values of the control parameter, wherein a value of the control parameter is selected based on the performance metric, 
   wherein the event detector, in the second operational mode, is configured to:
 receive a message stream for messages exchanged on the messaging platform in real-time; and 
   execute the event detection algorithm on the message stream according to the selected value of the control parameter to generate at least one second cluster chain.   
     
     
         2 . The messaging system of  claim 1 , wherein the control parameter includes a similarity threshold. 
     
     
         3 . The messaging system of  claim 1 , wherein the control parameter includes a resolution of a clustering algorithm. 
     
     
         4 . The messaging system of  claim 1 , wherein the performance metric includes a consolidation metric, the consolidation metric being defined by a ratio of a number of related entity pairs and a number of related entity pairs that share a common cluster chain. 
     
     
         5 . The messaging system of  claim 1 , wherein the performance metric includes a discrimination metric, the discrimination metric being defined by a ratio of a number of unrelated entity pairs and a number of unrelated entity pairs that are not in a common cluster chain. 
     
     
         6 . The messaging system of  claim 1 , wherein the event detector, in the first operational mode, is configured to compute a clustering score based on a consolidation metric and a discrimination metric, the performance metric including the clustering score. 
     
     
         7 . The messaging system of  claim 1 , wherein, to execute the event detection algorithm, the event detector, in the second operational mode, is configured to:
 identify a plurality of trending entities from the evaluation dataset stream;   compute similarity values based on frequency count and co-occurrences among the plurality of trending entities over a time window, each similarity value indicating a level of similarity between two trending entities;   generate a similarity graph based on the similarity values, the similarity graph including nodes representing the plurality of trending entities and edges being annotated with the similarity values; and   partition the similarity graph according to a clustering algorithm to detect one or more cluster groups of the at least one first cluster chain.   
     
     
         8 . The messaging system of  claim 7 , wherein the control parameter includes a similarity threshold, the similarity threshold being used in the second operational mode to filter the similarity graph. 
     
     
         9 . A method comprising:
 executing, in a first operational mode, an event detection algorithm on at least a portion of an evaluation dataset stream to generate at least one first cluster chain for varying values of a control parameter, the evaluation dataset stream including messages exchanged on a messaging platform;   computing, in the first operational mode, a performance metric regarding the execution of the event detection algorithm for the varying values of the control parameter, wherein a value of the control parameter is selected based on the performance metric;   receiving, in a second operational mode, a message stream for messages exchanged on the messaging platform in real-time; and   executing, in the second operational mode, the event detection algorithm on the message stream according to the selected value of the control parameter to generate at least one second cluster chain.   
     
     
         10 . The method of  claim 9 , wherein the control parameter includes a similarity threshold. 
     
     
         11 . The method of  claim 9 , wherein the control parameter includes a
 resolution of a clustering algorithm.   
     
     
         12 . The method of  claim 9 , wherein the performance metric includes a consolidation metric, the consolidation metric being defined by a ratio of a number of related entity pairs and a number of related entity pairs that share a common cluster chain. 
     
     
         13 . The method of  claim 9 , wherein the performance metric includes a discrimination metric, the discrimination metric being defined by a ratio of a number of unrelated entity pairs and a number of unrelated entity pairs that are not in a common cluster chain. 
     
     
         14 . The method of  claim 9 , further comprising:
 computing, in the first operational mode, a clustering score based on a consolidation metric and a discrimination metric, the performance metric including the clustering score.   
     
     
         15 . The method of  claim 9 , further comprising:
 identifying, in the second operational mode, a plurality of trending entities from messages posted to the messaging platform, wherein a trending entity is an entity mentioned by users in the messages posted to the messaging platform at a rate higher than other entities;   executing, in the second operational mode, one or more similarity-based clustering operations on the plurality of trending entities to detect a first cluster group of trending entities over a first period of time and a second cluster group of trending entities over a second period of time;   determining, in the second operational mode, that the second cluster group is related to the first cluster group;   generating, in the second operational mode, the at least one second cluster chain by linking the second cluster group with the first cluster group, the at least one second cluster chain representing an event over the first and second periods of time; and   storing, in the second operational mode, the at least one second cluster chain in a memory device on the messaging platform.   
     
     
         16 . A non-transitory computer-readable medium storing executable instructions that when executed by at least one processor cause the at least one processor to execute operations, the operations comprise:
 executing, in a first operational mode, an event detection algorithm on at least a portion of an evaluation dataset stream to generate at least one first cluster chain for varying values of a control parameter, the evaluation dataset stream including messages exchanged on a messaging platform;   computing, in the first operational mode, a performance metric regarding the execution of the event detection algorithm for the varying values of the control parameter, wherein a value of the control parameter is selected based on the performance metric;   receiving, in a second operational mode, a message stream for messages exchanged on the messaging platform in real-time; and   executing, in the second operational mode, the event detection algorithm on the message stream according to the selected value of the control parameter to generate at least one second cluster chain.   
     
     
         17 . The non-transitory computer-readable medium of  claim 16 , wherein the control parameter includes a similarity threshold. 
     
     
         18 . The non-transitory computer-readable medium of  claim 16 , wherein the control parameter includes a resolution of a clustering algorithm. 
     
     
         19 . The non-transitory computer-readable medium of  claim 16 , wherein the performance metric includes a consolidation metric, the consolidation metric being defined by a ratio of a number of related entity pairs and a number of related entity pairs that share a common cluster chain. 
     
     
         20 . The non-transitory computer-readable medium of  claim 16 , wherein the performance metric includes a discrimination metric, the discrimination metric being defined by a ratio of a number of unrelated entity pairs and a number of unrelated entity pairs that are not in a common cluster chain.