Part of Advances in Neural Information Processing Systems 11 (NIPS 1998)
Daniel Lee, Haim Sompolinsky
A directed generative model for binary data using a small number of hidden continuous units is investigated. A clipping nonlinear(cid:173) ity distinguishes the model from conventional principal components analysis. The relationships between the correlations of the underly(cid:173) ing continuous Gaussian variables and the binary output variables are utilized to learn the appropriate weights of the network. The advantages of this approach are illustrated on a translationally in(cid:173) variant binary distribution and on handwritten digit images.