Part of Advances in Neural Information Processing Systems 13 (NIPS 2000)
Tong Zhang
In theory, the Winnow multiplicative update has certain advantages over the Perceptron additive update when there are many irrelevant attributes. Recently, there has been much effort on enhancing the Perceptron algo(cid:173) rithm by using regularization, leading to a class of linear classification methods called support vector machines. Similarly, it is also possible to apply the regularization idea to the Winnow algorithm, which gives meth(cid:173) ods we call regularized Winnows. We show that the resulting methods compare with the basic Winnows in a similar way that a support vector machine compares with the Perceptron. We investigate algorithmic is(cid:173) sues and learning properties of the derived methods. Some experimental results will also be provided to illustrate different methods.