Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)
Matthias Burger, Thore Graepel, Klaus Obermayer
We derive and analyse robust optimization schemes for noisy vector quantization on the basis of deterministic annealing. Starting from a cost function for central clustering that incorporates distortions from channel noise we develop a soft topographic vector quantization al(cid:173) gorithm (STVQ) which is based on the maximum entropy principle and which performs a maximum-likelihood estimate in an expectation(cid:173) maximization (EM) fashion. Annealing in the temperature parameter f3 leads to phase transitions in the existing code vector representation dur(cid:173) ing the cooling process for which we calculate critical temperatures and modes as a function of eigenvectors and eigenvalues of the covariance matrix of the data and the transition matrix of the channel noise. A whole family of vector quantization algorithms is derived from STVQ, among them a deterministic annealing scheme for Kohonen's self-organizing map (SOM). This algorithm, which we call SSOM, is then applied to vector quantization of image data to be sent via a noisy binary symmetric channel. The algorithm's performance is compared to those of LBG and STVQ. While it is naturally superior to LBG, which does not take into account channel noise, its results compare very well to those of STVQ, which is computationally much more demanding.