On Sparsity and Overcompleteness in Image Models

Part of Advances in Neural Information Processing Systems 20 (NIPS 2007)

Bibtex »Metadata »Paper »

Authors

Pietro Berkes, Richard Turner, Maneesh Sahani

Abstract

<p>Computational models of visual cortex, and in particular those based on sparse coding, have enjoyed much recent attention. Despite this currency, the question of how sparse or how over-complete a sparse representation should be, has gone without principled answer. Here, we use Bayesian model-selection methods to address these questions for a sparse-coding model based on a Student-t prior. Having validated our methods on toy data, we find that natural images are indeed best modelled by extremely sparse distributions; although for the Student-t prior, the associated optimal basis size is only modestly overcomplete.</p>