Part of Advances in Neural Information Processing Systems 2 (NIPS 1989)
Anders Krogh, C. Thorbergsson, John Hertz
We introduce a cost function for learning in feed-forward neural networks which is an explicit function of the internal representa(cid:173) tion in addition to the weights. The learning problem can then be formulated as two simple perceptrons and a search for internal representations. Back-propagation is recovered as a limit. The frequency of successful solutions is better for this algorithm than for back-propagation when weights and hidden units are updated on the same timescale i.e. once every learning step.