Part of Advances in Neural Information Processing Systems 11 (NIPS 1998)
Sophie Denève, Alexandre Pouget, Peter Latham
Gain control by divisive inhibition, a.k.a. divisive normalization, has been proposed to be a general mechanism throughout the vi(cid:173) sual cortex. We explore in this study the statistical properties of this normalization in the presence of noise. Using simulations, we show that divisive normalization is a close approximation to a maximum likelihood estimator, which, in the context of population coding, is the same as an ideal observer. We also demonstrate ana(cid:173) lytically that this is a general property of a large class of nonlinear recurrent networks with line attractors. Our work suggests that divisive normalization plays a critical role in noise filtering, and that every cortical layer may be an ideal observer of the activity in the preceding layer.
Information processing in the cortex is often formalized as a sequence of a linear stages followed by a nonlinearity. In the visual cortex, the nonlinearity is best de(cid:173) scribed by squaring combined with a divisive pooling of local activities. The divisive part of the nonlinearity has been extensively studied by Heeger and colleagues [1], and several authors have explored the role of this normalization in the computation of high order visual features such as orientation of edges or first and second order motion[ 4]. We show in this paper that divisive normalization can also playa role in noise filtering. More specifically, we demonstrate through simulations that networks implementing this normalization come close to performing maximum likelihood es(cid:173) timation. We then demonstrate analytically that the ability to perform maximum likelihood estimation, and thus efficiently extract information from a population of noisy neurons, is a property exhibited by a large class of networks.
Maximum likelihood estimation is a framework commonly used in the theory of ideal observers. A recent example comes from the work of Itti et al., 1998, who have shown that it is possible to account for the behavior of human subjects in simple discrimination tasks. Their model comprised two distinct stages: 1) a network
Divisive Normalization. Line Attractor Networks and Ideal Observers
105
which models the noisy response of neurons with tuning curves to orientation and spatial frequency combined with divisive normalization, and 2) an ideal observer (a maximum likelihood estimator) to read out the population activity of the network.
Our work suggests that there is no need to distinguish between these two stages, since, as we will show, divisive normalization comes close to providing a maximum likelihood estimation. More generally, we propose that there may not be any part of the cortex that acts as an ideal observer for patterns of activity in sensory areas but, instead, that each cortical layer acts as an ideal observer of the activity in the preceding layer.