Part of Advances in Neural Information Processing Systems 15 (NIPS 2002)
Pascal Vincent, Yoshua Bengio
The similarity between objects is a fundamental element of many learn- ing algorithms. Most non-parametric methods take this similarity to be fixed, but much recent work has shown the advantages of learning it, in particular to exploit the local invariances in the data or to capture the possibly non-linear manifold on which most of the data lies. We propose a new non-parametric kernel density estimation method which captures the local structure of an underlying manifold through the leading eigen- vectors of regularized local covariance matrices. Experiments in density estimation show significant improvements with respect to Parzen density estimators. The density estimators can also be used within Bayes classi- fiers, yielding classification rates similar to SVMs and much superior to the Parzen classifier.