{"title": "Stability and Observability", "book": "Advances in Neural Information Processing Systems", "page_first": 1171, "page_last": 1172, "abstract": null, "full_text": "Stability and Observability \n\nMax Garzon \n\nFernanda Botelho \n\ngarzonmGhermea.maci.memat.edu botelhofGhermea.maci.memat.edu \nInstitute for Intelligent Systems Department of Mathematical Sciences \n\nMemphis State University \nMemphis, TN 38152 U.S.A. \n\nThe theme was the effect of perturbations of the defining parameters of a neural net(cid:173)\nwork due to: 1) mea\"urement\" (particularly with analog networks); 2) di\"cretization \ndue to a) digital implementation of analog nets; b) bounded-precision implementa(cid:173)\ntion of digital networks; or c) inaccurate evaluation of the transfer function(s}; 3) \nnoise in or incomplete input and/or output of the net or individual cells (particu(cid:173)\nlarly with analog networks). \n\nThe workshop presentations address these problems in various ways. Some develop \nmodels to understand the influence of errors/perturbation in the output, learning \nand general behavior of the net (probabilistic in Piche and TresPi optimisation in \nRojas; dynamical systems in Botelho k Garson). Others attempt to identify de(cid:173)\nsirable properties that are to be preserved by neural network solutions (equilibria \nunder faster convergence in Peterfreund & Baram; decision regions in Cohen). Of \nparticular interest is to develop networks that compute robustly, in the sense that \nsmall perturbations of their parameters do not affect their dynamical and observ(cid:173)\nable behavior (stability in biological networks in Chauvet & Chauvet; oscillation \nstability in learning in Rojas; hysterectic finite-state machine simulation in Casey). \nIn particular, understand how biological networks cope with uncertainty and errors \n(Chauvet & Chauvet) through the type of stability that they exhibit. \n\nQUESTIONS AND ANSWERS \n\nSome questions served to focus the presentations and discussion. Some were (par(cid:173)\ntially) answered, and others were barely touched: \n<> What are the mod \"ignificant error\" in defining parameter\" with re\"pect to output \nbehavior? By evidence presented, i/o and weights seem to be the most sensitive. \n<> Is there an essential difference between perturbations in weights (long-term mem(cid:173)\nory) and inputs (short-memory)? They seem to playa symmetric role in feedforward \nand, to some extent, recurrent nets. But evidence is not conclusive. \n<> How can the effects of perturbation\" be kept under control or eliminated altogether'! \nIf one is only interested in dynamical qualitative features, small enough errors of \nany kind (as incurred in digital implementations for example) are not relevant for \nmost nets (What you see on the screen is what should be happening). \n<> Are they architecture (in)dependentf On the other hand, they spread rapidly un(cid:173)\nder iteration and exact quantification varies with the architecture. \n<> Are stability and implementation based on dynamical features the only ways to \n\n1171 \n\n\f1172 \n\nGarzon and Botelho \n\ncope with error!/perturbatiofU f The difficulty to quantify (perhaps due to lack of \nresearch) seems to indicate so. Stability worth a closer look for its own sake. \n<> Doe, requiring robud computation really redrict the capabilitie, of neural net(cid:173)\nwork, f Apparently not, since in all likelihood there exist universal neural nets \nwhich tolerate small errors (see talk by Botelho & Garlon). Wide open. \n\nTALKS AND SHORT ABSTRACTS \n\n\u2022 TraJ~tory Control of Convergent Networks, Natan Peterfreund and Y. \nBaram. We present a class of feedback control functions which accelerate con(cid:173)\nvergence rates of autonomous nonlinear dynamical systems such as neural network \nmodels, without affecting the basic convergence properties (e.g. equilibrium points). \nnatanOtx.technion.ac.il \n\u2022 Sensitivity of Neural Network to Errors, Steven Piche. Using stochastic \nmodels, analytic expressions for the effects of such errors are derived for arbitrary \nfeedforward neural networks. Both, the degree of nonlinearity and the relationship \nbetween input correlation and the weight vectors, are found to be important in \ndetermining the effects of errors. picheOlllcc. COm \n\u2022 Stability of Learning in Neural Networks, Raul Roja!. Finding optimal \ncombinations of learning and momentum rates for the standard backpropagation \ninvolves difficult tradeoffs across fractal boundaries. We show that statistic prepro(cid:173)\ncessing can bring error functions under control. rOjaaOinf. fu-berlin.de \n\u2022 Stability of Purklnje Cells in Cerebellar Cortex, Gilbert Ohauvet and Pierre \nOhauvet. The cerebellar cortex (involved in learning and retrieving) is a hierarchical \nfunctional unit built around a Purkinje cell, which has its own functional proper(cid:173)\nties. We have shown experimentally that Purkinje dynamical systems have a unique \nsolution, which is asymptotically stable. It seems possible to give a general expla(cid:173)\nnation of stability in biological systems. chauvetOibt. uni v-angers. fr. \n\u2022 Recall and Learning with Deficient Data, Volker Tresp, Subutai Ahmad, \nRalph Neuneier. Mean values and maximum likelihood estimators are not the best \nways to cope with noisy data. See their LA:5 poster summary in these proceedings \nfor an extended abstract. treapOzfe. aiemena. de \n\u2022 Computation Dynamics in Discrete-Time Recurrent Networks, Mike \nOasey. We consider training recurrent higher-order neural networks to recognize \nregular languages, using the cycles in their diagrams for hysterectic simulation of \nfinite state machines. The latter suggests a general logical approach to solving the \n'neural code' problem for living organisms, necessary for understanding information \nprocessing in the nervous system. mcaseyOsdcc. ucsd. edu \n\u2022 Synthesis of Decision Regions in Dynamical Systems, Mike Oohen. As \na first step toward a representation theory of decision functions via neural nets, \nhe presented a method which enables the construction of a system of differential \nequations exhibiting a given finite set of decision regions and equilibria with a very \nlarge class of indices consistent with the Morse inequalities. mikeOpark. bu. edu \n\u2022 Observability of Discrete and Analog Networks, F. Botelho and M. Garzon. \nWe show that most networks (with finitely many analog or infinitely many boolean \nneurons) are observable (i.e., all their corrupted pseudo-orbits actually reflect true \norbits). See their DS:2 poster summary in these proceedings. \n\n\f", "award": [], "sourceid": 734, "authors": [{"given_name": "Max", "family_name": "Garzon", "institution": null}, {"given_name": "Fernanda", "family_name": "Botelho", "institution": null}]}