Twitter Anomaly Detection

Twitter’s user metrics rise and fall on daily and weekly cycles, and those cycles hide the anomalies that matter. S-H-ESD removes the seasonal and trend components and then robustly detects outliers in the residual, so a spike at three in the morning is judged against other three-in-the-mornings rather than against the week. It ran across tens of thousands of metrics — scaling infrastructure to load, catching spam accounts and abuse — and was open-sourced and made available on Twitter’s GitHub.

2013.