Design of Experiments via Information Theory

Part of Advances in Neural Information Processing Systems 16 (NIPS 2003)

Bibtex Metadata Paper


Liam Paninski


We discuss an idea for collecting data in a relatively efficient manner. Our point of view is Bayesian and information-theoretic: on any given trial, we want to adaptively choose the input in such a way that the mutual in- formation between the (unknown) state of the system and the (stochastic) output is maximal, given any prior information (including data collected on any previous trials). We prove a theorem that quantifies the effective- ness of this strategy and give a few illustrative examples comparing the performance of this adaptive technique to that of the more usual nonadap- tive experimental design. For example, we are able to explicitly calculate the asymptotic relative efficiency of the “staircase method” widely em- ployed in psychophysics research, and to demonstrate the dependence of this efficiency on the form of the “psychometric function” underlying the output responses.