Topic clustering and event detection

Abstract

A device for identifying events based on a dataset comprising datapoints. The device comprises a clustering module configured to cluster the dataset into topic clusters; and a cluster cohesion filter module configured to filter datapoints in each topic cluster based on each datapoint's datapoint-to-cluster distance.

The invention claimed is: 
     
         1 . A device for identifying event(s) based on documents comprising terms, the device comprising:
 a processor;   a non-transitory computer readable memory storing software code which, when executed on the processor, causes the processor:
 to extract the terms from the documents; 
 to cluster the documents into topic clusters based on a term frequency-inverse document frequency (TF-IDF) calculated from the extracted terms; 
 to filter documents in each topic cluster based on each document's document-to-cluster distance; and 
   a temporal feature module configured to determine, based on a temporal feature of the topic cluster, whether each topic cluster is related to an event and to determine whether each topic cluster is related to an event by determining a time property for each document and determining the number of documents that have their time property within a time window.   
     
     
         2 . The device according to  claim 1 , wherein the device is configured to:
 provide information indicative of any identified event(s).   
     
     
         3 . The device according to  claim 1 , wherein the device is configured to:
 receive a user provided corpus of works; and   match user provided work(s) relating to any identified event(s).   
     
     
         4 . The device according to  claim 3 , wherein the device is configured to:
 submit the matching work(s) to a social media account of the user on behalf of the user.   
     
     
         5 . The device according to  claim 1 , wherein the cluster cohesion filter module is configured to filter out a document if the document's document-to-cluster distance is over a threshold. 
     
     
         6 . The device according to  claim 5 , wherein the threshold is based on the document with the next least document-to-cluster distance. 
     
     
         7 . The device according to  claim 6 , wherein the threshold is based on the document-to-cluster distance of the document with the next least document-to-cluster distance. 
     
     
         8 . The device according to  claim 7 , wherein the threshold is equal to the document-to-cluster distance of the document with the next least document-to-cluster distance multiplied by a threshold multiplier. 
     
     
         9 . The device  according to 1 , wherein the cluster cohesion filter module is further configured to:
 sort the documents by their document-to-cluster distance.   
     
     
         10 . The device according to  claim 9 , wherein the cluster cohesion filter is further configured to compare adjacent documents in the sorted set of documents and if the difference in document-to-cluster distances between the adjacent points is over a threshold then the document with a higher document-to-cluster distance is filtered out of the cluster. 
     
     
         11 . The device according to  claim 10 , wherein the threshold is between 5 percent and 20 percent of the lower valued document-to-cluster distance. 
     
     
         12 . The device according to  claim 11 , wherein determining whether each topic cluster is related to an event based on a temporal feature of the topic cluster comprises is based on whether a number of documents in the time window is above a threshold amount. 
     
     
         13 . The device according to  claim 12 , wherein the threshold amount is between 60 percent and 95 percent of the topic cluster size. 
     
     
         14 . The device according to  claim 13 , wherein the time window is between 2 and 48 hours long. 
     
     
         15 . The device according to  claim 1 , wherein the event is a breaking news event. 
     
     
         16 . The device according to  claim 1 , wherein the documents are news articles.