Time makes sense: Event Discovery in Twitter using Temporal Similarity

Temporal text mining (TTM) has recently attracted the attention of scientists as a mean to discover and track in real-time discussions in micro-blogs. However current approaches to temporal mining suffer from efficiency problems when applied to large micro-blog streams, like Twitter, now reaching an average of 500 million tweets per day. We propose a technique, named SAX (based on an algorithm named Symbolic Aggregate Approximation) to discretize the temporal series of terms into a small set of levels, leading to a string for each terms. We then define a subset of "interesting" strings, i.e. Those representing patterns of collective attention. Sliding temporal windows are used to detect clusters of terms with the same string. We show that SAX is more efficient (by orders of magnitude) than other approaches to temporal mining in literature. In this paper, we experiment SAX on the task of event discovery over one year 1% world while Twitter stream.