Algorithm predicts trending topics on Twitter, in advance!

When the tweets about any topic explode - twitter lists them as 'trending topics'. Those who get on the top of these trending topics get a ton of free publicity. Twitter's got a sophisticated algorithm that factors the overall number of tweets about any particular topic and the increase in the number over a short span of time to rank the topics in the trends. However MIT's engineers have a way to predict which topics will trend on Twitter well in advance. The best part? Their algorithm is 95% accurate!



MIT's associate professor Devavrat Shah and his student Stanislav Nikolov will present their algorithm at the "Interdisciplinary Workshop on Information and Decision in Social Networks" in this month. The algorithm finds out which topics will trend at an interval of about 90 minutes. The duo say that they can predict the trending topics almost about 4-5 hours in advance. But like all the current generation artificially intelligent computer systems, this new algorithm needs to be trained. It creates samples of the data and analyses the patterns. However, it does not rely on the shape of the patterns to make predictions.

Shah says that their algorithm compares the change in the number of tweets about any particular topic to the sample set it has collected. Each sample then 'votes' for the possibility of the new topic which may trend but some of the samples are given more weight in deciding the final outcome. The algorithm has training set of about 200 twitter topics that did trend and another set of 200 topics that didn't trend. Experiments have demonstrated about 95% success rate and ~4% false positive rate. Quite impressive!

We won't be surprised to see Twitter officials being interested in this algorithm. Twitter could use this in a number of ways including charging a premium to the advertisers to bank on the topics that will trend in the next few hours.

Via: MIT