We discuss the application of TAP mean field methods known from the Statistical Mechanics of disordered systems to Bayesian classifi(cid:173) cation models with Gaussian processes. In contrast to previous ap(cid:173) proaches, no knowledge about the distribution of inputs is needed. Simulation results for the Sonar data set are given.
1 Modeling with Gaussian Processes
Bayesian models which are based on Gaussian prior distributions on function spaces are promising non-parametric statistical tools. They have been recently introduced into the Neural Computation community (Neal 1996, Williams & Rasmussen 1996, Mackay 1997). To give their basic definition, we assume that the likelihood of the output or target variable T for a given input s E RN can be written in the form p(Tlh(s)) where h : RN --+ R is a priori assumed to be a Gaussian random field. If we assume fields with zero prior mean, the statistics of h is entirely defined by the second order correlations C(s, S') == E[h(s)h(S')], where E denotes expectations
310
M Opper and 0. Winther
with respect to the prior. Interesting examples are