Online Learning for Latent Dirichlet Allocation

Part of Advances in Neural Information Processing Systems 23 (NIPS 2010)

Bibtex »Metadata »Paper »Supplemental »

Authors

Matthew Hoffman, Francis Bach, David Blei

Abstract

<p>We develop an online variational Bayes (VB) algorithm for Latent Dirichlet Allocation (LDA). Online LDA is based on online stochastic optimization with a natural gradient step, which we show converges to a local optimum of the VB objective function. It can handily analyze massive document collections, including those arriving in a stream. We study the performance of online LDA in several ways, including by fitting a 100-topic topic model to 3.3M articles from Wikipedia in a single pass. We demonstrate that online LDA finds topic models as good or better than those found with batch VB, and in a fraction of the time.</p>