The Storage Capacity of a Fully-Connected Committee Machine

Part of Advances in Neural Information Processing Systems 10 (NIPS 1997)

Bibtex Metadata Paper

Authors

Yuansheng Xiong, Chulan Kwon, Jong-Hoon Oh

Abstract

We study the storage capacity of a fully-connected committee ma(cid:173) chine with a large number K of hidden nodes. The storage capac(cid:173) ity is obtained by analyzing the geometrical structure of the weight space related to the internal representation . By examining the as(cid:173) ymptotic behavior of order parameters in the limit of large K, the storage capacity Q c is found to be proportional to ]{ Jln ]{ up to the leading order. This result satisfies the mathematical bound given by Mitchison and Durbin , whereas the replica-symmetric solution in a conventional Gardner's approach violates this bound.