TAP Gibbs Free Energy, Belief Propagation and Sparsity

Part of Advances in Neural Information Processing Systems 14 (NIPS 2001)

Bibtex Metadata Paper


Lehel Csató, Manfred Opper, Ole Winther


The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with quadratic interactions and arbritary single site constraints is derived. We show how a specific sequential minimization of the free energy leads to a generalization of Minka’s expectation propa- gation. Lastly, we derive a sparse representation version of the sequential algorithm. The usefulness of the approach is demonstrated on classifica- tion and density estimation with Gaussian processes and on an indepen- dent component analysis problem.