{"title": "Hallucinations in Charles Bonnet Syndrome Induced by Homeostasis: a Deep Boltzmann Machine Model", "book": "Advances in Neural Information Processing Systems", "page_first": 2020, "page_last": 2028, "abstract": "The Charles Bonnet Syndrome (CBS) is characterized by complex vivid visual hallucinations in people with, primarily, eye diseases and no other neurological pathology. We present a Deep Boltzmann Machine model of CBS, exploring two core hypotheses: First, that the visual cortex learns a generative or predictive model of sensory input, thus explaining its capability to generate internal imagery. And second, that homeostatic mechanisms stabilize neuronal activity levels, leading to hallucinations being formed when input is lacking. We reproduce a variety of qualitative findings in CBS. We also introduce a modification to the DBM that allows us to model a possible role of acetylcholine in CBS as mediating the balance of feed-forward and feed-back processing. Our model might provide new insights into CBS and also demonstrates that generative frameworks are promising as hypothetical models of cortical learning and perception.", "full_text": "Hallucinations in Charles Bonnet Syndrome Induced\nby Homeostasis: a Deep Boltzmann Machine Model\n\nDavid P. Reichert, Peggy Series and Amos J. Storkey\n\n{d.p.reichert@sms., pseries@inf., a.storkey@} ed.ac.uk\n\nSchool of Informatics, University of Edinburgh\n\n10 Crichton Street, Edinburgh, EH8 9AB\n\nAbstract\n\nThe Charles Bonnet Syndrome (CBS) is characterized by complex vivid visual\nhallucinations in people with, primarily, eye diseases and no other neurological\npathology. We present a Deep Boltzmann Machine model of CBS, exploring\ntwo core hypotheses: First, that the visual cortex learns a generative or predic-\ntive model of sensory input, thus explaining its capability to generate internal\nimagery. And second, that homeostatic mechanisms stabilize neuronal activity\nlevels, leading to hallucinations being formed when input is lacking. We repro-\nduce a variety of qualitative \ufb01ndings in CBS. We also introduce a modi\ufb01cation to\nthe DBM that allows us to model a possible role of acetylcholine in CBS as me-\ndiating the balance of feed-forward and feed-back processing. Our model might\nprovide new insights into CBS and also demonstrates that generative frameworks\nare promising as hypothetical models of cortical learning and perception.\n\n1\n\nIntroduction\n\nComplex visual hallucinations [1] can offer a fascinating insight into how the brain realizes visual\nperception. The content of such hallucinations can be highly elaborate, consisting of people, ani-\nmals, objects and whole scenes, and the images supposedly can \u201cexceed anything seen in real life\u201d\nin detail and vividness [1]. Attempts have been made to unify complex hallucinations in various\npathologies in one qualitative model, but many argue that the underlying causal mechanisms are too\nvaried to do so [2]. Of particular interest is the Charles Bonnet Syndrome (CBS) [3, 4, 5], where\npatients experience complex visual hallucinations which appear not to be causally related to any\nother impairment to mental health and where the primary pathology is one of loss of vision due to\neye diseases. Sensory deprivation is thus implicated as playing a key role in the development of\nCBS, and comparisons have been made to phantom limb phenomena [3, 5].\nThe mechanisms behind complex hallucinations remain obscure. Theories of CBS are descriptive in\nnature and no computational model exists. For example, hallucinations are attributed to \u2018perceptual\ntraces being released\u2019 [3] that would normally be inhibited by sensory input, or it is suggested that\nexperience in general is evoked by internally generated neuronal activity in distributed networks,\na \u2018neuromatrix\u2019, imposing meaning on sensory input or onto unspeci\ufb01c input in the case of hallu-\ncinations [3, 5]. The phenomenon of internally generated images becomes less mysterious if one\nassumes the cortex implements an actual generative model of sensory input. The hypothesis that\ncortical learning is driven by prediction or reconstruction of sensory input is promising as it could\nexplain how the brain might learn in an unsupervised fashion, evaluating its internal model of the\nworld by matching predictions to actual input [6, 7]. Consequently, the idea that disorders includ-\ning hallucinations are caused by mismatches between internally generated expectations and sensory\ninput has recently found interest in psychology [8, 9]. If generating internal imagery is an essen-\ntial aspect of normal vision, then this could explain why hallucinations occur in so many different\npathologies, even sometimes when there is no direct malfunction of the visual system itself [1].\n\n1\n\n\fOne modeling framework that implements unsupervised generative learning in a neural architecture\nis the Deep Boltzmann Machine (DBM) [10]. DBMs have been developed in a machine learning\ncontext, but we argue that they could model aspects of cortical learning and perception as well. They\nare related to Hop\ufb01eld networks, which have been used in the context of models of hallucinations\nbefore [11]. However, whereas the latter model some abstract memory system, DBMs (and the\nrelated Deep Belief Nets) learn hierarchical representations of data [7], thus capturing aspects of the\nvisual cortex [12], the locus where visual hallucinations are ultimately realized [1, 13]. We aim to\nrelate inference in a DBM to mechanisms of cortical perception.\nWe thus present a DBM model of the CBS, and propose a concrete mechanism that could lead to\nhallucinations being formed: homeostasis. There is strong experimental evidence that homeostatic\nprocesses serve to stabilize the activity level of neuronal populations through a variety of cellular\nand synaptic mechanisms [14]. Moreover, deafferentiated cortex becomes hyper-excitable, and it\nhas been suggested before that this could be a result of homeostasis [15]. Hence, in CBS a lack\nof visual input could lead to intrinsic excitability changes of neurons setting in to restore original\nactivity levels. In our model, we demonstrate how these changes could cause spontaneous \u2018complex\u2019\nhallucinations to be formed even when sensory input is lacking. These hallucinations are complex\nin the sense that they involve learned, distributed representations of objects in (toy) images rather\nthan, for example, corresponding to local structural features of topographically organized cortical\nareas, the latter being implicated in simpler hallucinations such as geometric patterns [16].\nIn Section 3, we \ufb01rst show that homeostasis can be bene\ufb01cial in a DBM, enabling the model to\nrecover correct internal representations from degraded input. Then we move on to hallucinations.\nThe CBS is a complex phenomenon and differs considerably among patients, but we can qualita-\ntively reproduce several aspects found in most or some cases: An initial latent period after loss of\nvision that is free of hallucinations; a localization of hallucinations to lesioned parts of the visual\n\ufb01eld (Section 3.1), potentially also explaining a tendency to see hallucinated objects too small for\ntheir surroundings; and, effects of cortical lesions and cortical suppression of activity (Section 3.2).\nMoreover, hallucinations in CBS tend to occur more often in states of drowsiness, implicating a\nrole of cholinergic and serotonergic systems [1]. By introducing a modi\ufb01cation to the DBM model,\nwe can account for this by taking acetylcholine to modulate the bottom-up top-down balance of\ninformation \ufb02ow (Section 4). Finally, we speculate on the potential of the DBM to model other\npathologies and on the difference between hallucinations and mental imagery (Section 5).\n\n2 Deep Boltzmann Machines\n\nBoltzmann machines (BMs) [17] are closely related to Hop\ufb01eld networks, which have been em-\nployed as models of hallucinations before (e.g. [18]). Both models consist of neural units x (here\nbinary with value 1 or 0, on or off) connected with symmetric weights W. A unit\u2019s state is de-\ntermined by a sigmoid activation function, and biases b control the excitability of each unit. The\noverall state of the network evolves according to an \u2018energy\u2019 function, E(x) = \u2212xT Wx \u2212 bT x,\nwhere minima in the energy landscape correspond to attractor states. A BM differs from a Hop\ufb01eld\nnet in two important points. First, a BM is stochastic in that the activation of a unit determines the\nprobability for it to switch on:\n\nP (xi = 1|x) = \u03c3((cid:88)\n\nwijxj + bi) =\n\nj\n\n1 + exp(\u2212(cid:80)\n\n1\nj wijxj \u2212 bi) .\n\n(1)\n\nWhen the model is run by switching units on and off stochastically, it performs a random walk\nin the energy landscape. Asymptotically, any state x will be assumed with probability P (x) \u221d\nexp(\u2212E(x)). Hence, a BM can be understood as modeling the probability distribution of the data\nrather than just as a memory network, which is why these models are of interest in machine learning.\nThe second difference is the possible introduction of hidden units, separating x into visible units\nv and hidden units h. Whereas the former represent visible variables such as the pixels of an\nimage, the latter represent latent variables that help to explain the image. Learning in a BM is\nthen performed with the aim of forming hidden representations from which data can be gener-\nated/predicted/reconstructed. In modeling P (x) for any x, not just data seen in training, one goal is\nto learn latent representations that make it possible to generalize over novel data. Another goal is to\nlearn representations which can then be utilized further, for example for classi\ufb01cation or clustering\n\n2\n\n\fFigure 1:\nExample images\nfrom the data sets (blank set\nnot shown).\n(a): Training set.\n(b): Corrupted set.\n(c): Noise set.\n(d): Top half blank set.\n\n[19]. The modeling context of a BM is thus rather different from that of a Hop\ufb01eld network.\nA Deep Boltzmann Machine (DBM) [10] is a BM with a special architecture (Figure 2a) consisting\nof a visible layer and several subsequent hidden layers stacked on top of each other. To simplify\ncomputations there are no lateral connections within any layer. When trained on a data set of im-\nages, each pair of adjacent layers is trained one at a time so that each layer learns to generate the\nactivations of the layer below, using only biologically plausible local Hebbian (and anti-Hebbian)\nweight changes. See [10] for details.1 Furthermore, to make a more concrete connection to the\nvisual cortex, we impose a hierarchical receptive \ufb01eld structure on the model: Each layer\u2019s units are\narranged topographically, and each unit\u2019s weights are restricted so that it receives inputs only from\na square patch of units below. In detail, the model had 20x20 visible units corresponding to images\nwith 20x20 pixels, and three hidden layers of 26x26 units each. Each unit in the \ufb01rst hidden layer\nreceived inputs from a 7x7 patch of visible units, whereas the higher layers received inputs from\nhalf (13x13) and all (26x26) of the units of the respective lower layer. The training data set used\nconsisted of toy images (Figure 1a), containing individual shapes out of three categories (upwards\ntriangles, downward triangles, squares) at random positions.\n\n2.1 Sampling and decoding the internal state\n\nTo model perception, we clamp the visible units to an image and sample the hidden units, starting\nfrom the \ufb01rst hidden layer and proceeding to the topmost, then going downwards, and repeating this\ncycle. For each layer, all units can be sampled in parallel. Processing across the cortical hierarchy\nis suggested to be cyclic as well [20].\nWe are interested in the representations formed internally, in the hidden layers of the DBM, when\nvisual input is \ufb01xed or lacking in the case of CBS. To decode the states of the hidden layers, we de\ufb01ne\na top-down projection to obtain a reconstructed image. Given the states xk of the hidden layer k in\nquestion, the activations ak\u22121 of the layer below are computed taking only xk into account, ignoring\nthe states xk\u22122 further below.2 This process is repeated down to the visibles, so that we obtain a\nreconstructed image which has been determined only from the states in layer k (using activations\ninstead of samples to obtain less noisy images). Note that we perform the top-down projection only\nto inspect the internal states. For the actual inference procedure, all intermediate layers are always\nsampled properly taking both adjacent layers into account.3 In this work, hidden states are always\ninitialized to zero for each image and evaluated after 40 sampling cycles.\nTo evaluate the quality of an internal representation when a shape image is presented to the model,\nwe compute the maximum value of the normalized cross-correlation of the projected reconstruction\nwith that image. In the case of hallucinations, internal representations are matched against all three\ntemplate shapes, taking the one that matches best as being represented and the corresponding cross-\ncorrelation value as measure of the quality of the hallucination (Figure 2b).\n\n2.2 Homeostasis in a DBM\n\nHomeostatic mechanisms in the cortex are found to stabilize neuronal activity [14]. In our model, we\nassume that neurons have an individual preferred activation level that is attained as representations\nof inputs are learned. Hence, after the model has been trained we compute each unit\u2019s activation\naveraged over 40 sample cycles over all training images, taking this as the \u2018healthy\u2019 activation level\n\n1We used 1 step contrastive divergence for the greedy layer-wise training, which is an approximation to\nmaximum likelihood gradient ascent learning, and performed no further training of the full DBM. The training\nset consisted of 60,000 images split into mini-batches of 100 and was iterated over through 30 epochs.\n\n2To compensate for the lack of bottom-up input in this case, the weights are doubled.\n3For computing the projections we use the original biases, not affected by homeostasis.\n\n3\n\n(a)(b)(c)(d)\f(a) Model setup, perception & emerging hallucinations.\n\n(b) Hallucination qualities.\n\nFigure 2: (a) left-hand side: Setup of the model and perception. With the visible units (dark) clamped\nto an image (bottom left), the hidden layer states assume representations of that image. Displayed\nare the decoded projections for each layer after ten recurrent cycles (left column). (a) right-hand\nside: After homeostatic regulation given empty images, hallucinations form spontaneously. Hallu-\ncinations are often stable after a few tens of recurrent cycles, but still \ufb02uctuate due to the stochastic\nnature of the DBM. They are slightly less well formed in lower layers, which require higher layer\ninput to form stable shape perceptions. (b): Examples of hallucinations of different qualities (com-\nputed from cross-correlations with templates). 1.0 is perfect match with template.\nunder normal sensory input. To simulate CBS, we then blank the visual input and let the model\nemploy homeostatic regulation to recover healthy activation levels. The homeostatic mechanism is\nimplemented in a straightforward way, adjusting only the biases (as in [12]) to model changes to\nintrinsic excitability of a neuron. Speci\ufb01cally, we present a mini-batch of 100 (corrupted or blank)\nimages for 40 cycles each to compute each unit\u2019s new average activity ai, and with the preferred\nactivity pi modify the unit\u2019s bias bi according to4\n\n\u2206bi = \u03b7(pi \u2212 ai),\n\n(2)\nwhere \u03b7 is rate of change parameter (set to 0.1, but the precise value was not found to matter). This\nwas repeated for 1000 iterations, and \u2206b averaged over the population tended to zero before that\nas original activity levels were restored. We note that similar mechanisms have been employed in\nDBM-like models during training itself to enforce sparsity in the activations [12, 21], resulting in\nV1 and V2 like receptive \ufb01elds being learned [12]. However such sparsity is not focus of this paper.\n\n3 Hallucinations emerging due to homeostasis\n\nFirst, we demonstrate that homeostasis can be a bene\ufb01cial mechanism in a DBM. To this end, we\npresent the trained model with heavily corrupted training images (Figure 1b) in which pixels where\nturned off with probability 0.65, emulating pre-cortical damage to the visual input. We then com-\nputed the reconstruction quality (Section 2.1) from the reconstructions projected from the top hidden\nlayer states (after 40 cycles of sampling), and found it to be 0.46 on average. For comparison, av-\nerage reconstruction quality on the uncorrupted training images was 0.98. The degradation of input\nwas also re\ufb02ected in changes in mean activities of the layers (Figure 3a). We then applied homeosta-\nsis as described in Section 2.2. As the excitability of the units was adjusted, mean activity levels for\neach hidden layer were gradually restored. At the same time, the reconstruction quality rose to about\n0.9 (Figure 3a). Thus, a simple local activity stabilization can help alleviating damage to the system.\nNote that due to excitation and inhibition being mixed in the weights of a BM, a lack or degradation\nof input changes but not necessarily decreases activity. Homeostasis in a DBM thus restores activity\nlevels in some cases by increasing and in some cases by attenuating neuronal excitability.\nTo model the CBS which is often triggered by profound retinal damage, we then repeated the home-\nostasis experiment with blank images. Now, any formation of internal representations could be\nregarded as hallucinations. The question was whether internal representations were stable and corre-\nsponded to \u2018objects\u2019 the model had learned, rather than random patterns. After all, the local changes\nof excitability and the permanent loss of visual input could have interfered with the dynamics of in-\nternal representations. As shown in Figure 3b, the activity levels of the hidden layers were restored\n\n4Equation 2 is minimizing the cross entropy in between p and a [21].\n\n4\n\nperceptionhallucination10 cycles1 cycle10 cls20 cls40 cls0.90.80.70.4\f(a) Corrupted input.\n\n(b) Blank input.\n\nFigure 3: (a): For corrupted images, homeostatic restoration of original activity levels (left \ufb01gure,\ndashed lines) in the three hidden layers, and recovering reconstruction quality (right \ufb01gure). (b): For\nblank images, restoration of activity (left) and qualities of emerging hallucinations (right). Each dot\nmarks one hallucination, plotted for 25 trials out of 100 per iteration. Blue curve is average quality.\n\nby homeostasis. To analyze the internal representations of the model, we computed the qualities of\nindividual representations of the topmost hidden layer from the projected reconstructions (Figure\n3b). Over an initial period, internal representations correspond to blank images even though activity\nlevels gradually improve. At some point however, hallucinations emerge, and relatively soon can\nthey reach high quality levels. At this point activities start to change more rapidly, hence halluci-\nnations themselves contribute to the restoration of activity levels. These results are consistent with\nCBS: If loss of vision is abrupt, hallucinations emerge after an initial latent period lasting hours to\ndays [5], which matches well the time scale on which homeostatic mechanisms take place [14].\nBesides a peak of hallucination quality at 1.0, there are also numerous hallucinations of lower qual-\nity, which could be in line with CBS as there complex hallucinations are often mixed with simple,\nless sophisticated ones. Also, some of the lower quality hallucinations are of transitory nature (if\nrun for 200 instead of 40 cycles, mean quality rose from 0.83 to 0.88). Still, in Section 4 we will\npresent mechanisms that lead to more stable hallucinations.\nWe also repeated the experiment with images containing random noise instead of being blank (Figure\n1c) to simulate a different type of visual impairment than total blindness. We found in this case\nthat smaller overall bias shifts were necessary to restore original activity levels (not shown) and\nproduce hallucinations (Figure 5a). This shows that the exact nature of visual impairment could\nhave an impact on whether and when hallucinations are formed. Indeed, many CBS patients develop\nhallucinations as vision degrades, but stop hallucinating when vision is \ufb01nally lost completely [5].\nIn our model, this could be explained when one assumes there is a limit to how much neuronal\nexcitability can be adapted. Thus, as long as there is some input, even if it is unspeci\ufb01c noise,\nhallucinations can be formed, but losing the input completely might require too much of a bias\nshift. Another reason for the cessation of hallucinations over time could be input speci\ufb01c synaptic\nplasticity, i.e. learning. If we were to train the model on empty images, it would learn to generate\nthose. Hence, homeostasis as a short-term stabilization mechanism could lead to hallucinations, but\na long-term reorganization of the cortex to represent the novel input would cause them to cease.\n\n3.1 Localized hallucinations with localized lesions\n\nAnother property of the hallucinations was that the represented shapes were found to be distributed\nover the whole image and could be any of the three categories (Figure 4a). This is of relevance as\ncomplex hallucinations in CBS vary from episode to episode in a majority of patients [4]. Hence, it\nis important that the model can form a variety of internal representations of learned images instead\nof just converging to a few degenerate states.\nDamage to the visual input leading to CBS can be restricted to parts of the visual \ufb01eld. Some\nstudies [5] report that hallucinations tend to be localized to the blind regions. To test whether we\ncould reproduce this, we repeated the homeostasis experiment with images from the training data set\nin which only the top half had been blanked out (Figure 1d), simulating a localized impairment of\nvision. As before, we found mean activities initially to be changed due to the degraded input and then\nbe restored after homeostatic regulation. To test whether hallucinations would form at any location,\nwe then tested the model on blank images. As shown in Figure 4b, the stable hallucinations were\n\n5\n\n0200400600iteration0.10.2mean activitylayer 1layer 2layer 30200400600iteration0.30.40.50.60.70.80.91.0reconstruction quality0200400600iteration0.10.2mean activitylayer 1layer 2layer 30200400600iteration0.30.40.50.60.70.80.91.0hallucination quality\f(a) Full lesion.\n\n(b) Top half lesion.\n\n(c) Right half lesion.\n\nFigure 4: Localization (center of mass) of high quality (>0.95) hallucinations in projected images at\nthe end of homeostatic regulation. Counts in thousands out of 105 trials. (a): Although local hotspots\nexist and squares are least likely to occur, overall hallucinations vary in type and location. (b): When\nonly the top half of visual input was blanked during homeostatic regulation, hallucinations emerged\n(c): When a region too narrow for triangles was blanked instead,\nlocalized to the \u2018blind\u2019 half.\nhallucinations were almost always squares.\n\nlocated in the top half of the visual \ufb01eld only, corresponding to the region of the \u2018lesion\u2019. Excitability\nchanges in the network are thus speci\ufb01c enough to have topographic properties.\nThis result lets us speculate on another phenomenon of CBS. In some cases, hallucinated objects\nare seen as too small for their surroundings (\u201cLilliputian\u201d)[5]. If hallucinations are constrained by a\nblind area of restricted size, and there is a tendency to see whole objects (rather than parts), then this\nwould mean that hallucinated objects would have to be small simply to \ufb01t into the blind area, often\ntoo small to \ufb01t the real surroundings (e.g. a tiny hallucinated person in a real room). We can test this\nin our model: In the training data set, the square shape is less wide than the triangle shapes (Figure\n1a). We repeated the last homeostasis experiment with the right half lesioned (9 pixels wide) instead\nof the top half, meaning that now only a (hallucinated) square would \ufb01t into the blind area. Indeed,\nwe found that now, stable hallucinations are mostly squares by a large margin (Figure 4c), despite\nthe fact that for fully blank images and for top lesioned images, squares were by far less common.5\nThe network thus relied on hallucinations that \ufb01tted the blind region to restore its activity levels.\n\n3.2 Cortical damage and suppression\n\nDamage to the visual system causing complex hallucinations can also be cortical, e.g. resulting from\nstroke. According to [1], such stroke damage needs to be located in earlier visual areas, whereas\nit is higher, associative areas that are argued to be both necessary and suf\ufb01cient for complex hal-\nlucinations. The interpretation is thus that the lack of bottom-up input somehow \u2018releases\u2019 activity\nin higher areas. However, in [22] transcranial magnetic stimulation was applied to early cortical\nareas of a CBS patient in a way thought to suppress cortical excitability. This led to a cessation of\nhallucinations. The authors point out that this \ufb01nding contradicts the release theory. Hence, if the\ninitial loss of vision is caused by damage to early cortical areas, complex hallucinations can form\nover time. If on the other hand activation in early areas is suppressed when CBS symptoms have\nalready been developed due to for example eye disease, hallucinations cease, at least temporarily.\nWe reproduced these \ufb01ndings. Taking the \ufb01rst hidden layer as representing an early cortical area,\nwe repeat the homeostasis experiment with the hidden units in that layer clamped to activations\nas if they were receiving no input (instead of clamping the visibles to blank images), simulating a\ncortical lesion. Again we found stable hallucinations to emerge in the higher layers (not shown).\nThen, to simulate the temporary suppression experiment in [22], we take the original model that\nhad homeostatic regulation applied with all hidden layers intact and developed hallucinations in the\nprocess, and clamp the \ufb01rst hidden layer to see whether that would interfere with already established\nhallucinations. We found that indeed, hallucinations ceased.\nWe note that due to the hierarchical receptive \ufb01eld structure in the model, the topmost hidden layer\nplays a special role, its units having the largest receptive \ufb01elds. We \ufb01nd that a DBM trained without\n\n5Square hallucinations were found to be least common after homeostasis over several model instances,\nalthough this bias did not exist in generations from the original models. This shows that the homeostasis\ninduced model is not equivalent to the original one. While intriguing, we did not examine this effect further.\n\n6\n\n05101520image row0.00.51.01.52.02.53.03.5hallucination count (1000)05101520image row0246810hallucination count (1000)trngle-uptrngle-dnsquare05101520image column024681012hallucination count (1000)\fFigure 5: Comparison of hallucination emer-\ngence for blank vs. noise images (a) or various\nvalues of ACh balance factor \u03b1 (b). Mean qual-\nities are plotted against the total homeostatic\nadaptation, de\ufb01ned as the absolute change of\nbiases averaged over all units. Both noise and\nlow \u03b1 lower the amount of adaptation neces-\nsary to elicit hallucinations. Low \u03b1 also in-\ncreases average hallucination quality.\n\n(a) Blank vs. noise images.\n\n(b) ACh levels.\n\nthe topmost layer failed to learn a generative model of the toy shapes. Thus, the top-most layer or pair\nof layers is necessary for generating complex hallucinations, as is the case with the higher associative\nvisual areas (although processing in the cortical hierarchy is of course much more complicated than\nin our model). They are also suf\ufb01cient in our model in so far as homeostasis did induce hallucinations\nas long as the \ufb01rst hidden layer was clamped at the outset. However, as the last experiment has\nshown, when hallucinations emerge due to activity changes in the whole system, then interfering\neven with a lower layer can disrupt their formation.\nWe also argue that this is not merely a matter of the higher layers lacking unspeci\ufb01c input from\nlower layers. When hallucinations are formed they evoke corresponding representations in all hidden\nlayers, even the lower ones that by themselves cannot support stable shape representations (Figure\n2b). Thus, this is a result of recurrent interaction with the higher layers, likely contributing to\nthe stability of the overall internal state. Clamping the \ufb01rst hidden layer prevents such recurrent\nstabilization. A role of recurrent interactions for hallucinations is also suggested in [23].\n\n4 A novel model of acetylcholine and its role in CBS\n\nCBS hallucinations are more likely to occur in states of drowsiness [1, 5]. This suggests a role\nof cholinergic and serotonergic systems, which in turn are implicated in pathologies of complex\nhallucinations other than CBS as well [1]. There is experimental evidence that acetylcholine (ACh)\nacts speci\ufb01cally to emphasize sensory input over internally generated one, mediating \u201cthe switching\nof the cortical processing mode from an intracortical to an input-processing mode\u201d [24]. Similarly,\nACh has been modeled to modulate the interaction in between bottom-up and top-down processing\n[25], the former delivering sensory information, the latter prior expectations.\nWe present a new model of ACh in the DBM framework. We take the notion that ACh in\ufb02uences\nthe balance of bottom-up and top-down one step further, suggesting that in the hierarchical cortex\nconsisting of several processing stages, ACh could mediate this balance at any stage. In the DBM\nmodel, each (intermediate) hidden layer receives input from a layer below, conveying sensory in-\nformation, and from a layer above that has learned to generate or predict the former layer\u2019s activity.\nWe thus take ACh to set the balance in between feed-forward and feed-back \ufb02ow of information. To\nthis end, we introduce a balance factor \u03b1\u0001[0, 1] so that an intermediate layer x(k) is sampled as\n\ni = 1|x(k\u22121), x(k+1)) = \u03c3((cid:88)\n\nP (x(k)\n\n+(cid:88)\n\n2\u03b1w(k\u22121)\n\nji\n\nx(k\u22121)\n\nj\n\n2(1 \u2212 \u03b1)w(k+1)\n\nij\n\nx(k+1)\nj\n\n),\n\n(3)\n\nj\n\nj\n\ngiven states x(.) and weights W(.) above and below (biases omitted for brevity). Hence, \u03b1 = 1\nequals maximal feed-forward \ufb02ow of information, and \u03b1 = 0.5 recovers the normal sampling mode.\nWe model the effect of drowsiness on hallucinations in CBS as follows: We assume that drowsiness\nis being re\ufb02ected as a decrease in ACh, modeled as \u03b1 < 0.5 in both intermediate hidden layers.\nAs states of drowsiness are intermittent with periods of normal or increased vigilance (there is no\npathology of these aspects in CBS per se), we assume that on average, ACh levels are still balanced.\nHence, we repeat the original homeostasis experiment with bias shifts determined with \u03b1 = 0.5, but\nat regular intervals test the model with \u03b1 = 0.3, re\ufb02ecting temporary phases of drowsiness.\nResults are displayed in Figure 5b. We \ufb01nd that with decreased levels of ACh, not only is a much\nsmaller homeostatic shift of excitability necessary to elicit hallucinations, but the average hallucina-\ntion quality is also superior. For example, at a mean bias shift of 0.5, mean hallucination quality with\n\u03b1 = 0.3 is already much higher than with \u03b1 = 0.5 at maximal bias shift, whereas hallucinations\n\n7\n\n0.00.20.40.60.81.0total homeost. adaptation0.30.40.50.60.70.80.91.0mean hallucination qualityblanknoise0.00.20.40.60.81.0total homeost. adaptation0.30.40.50.60.70.80.91.0mean hallucination quality0.70.50.3\fat balanced ACh levels have not even emerged yet at this point. This would thus correspond to a\nsituation where hallucinations would only occur during drowsiness. For comparison, we also did\nthe tests with an increased ACh level of \u03b1 = 0.7. In that case, hallucinations never emerge over the\ncourse of the homeostatic process (the end of which is determined from activities computed with\n\u03b1 = 0.5). In summary, we found that a temporary change in the balance of feed-forward and feed-\nback \ufb02ow of information can have a profound effect on the emergence of hallucinations, yielding a\npotential explanation for the role of drowsiness and ACh in CBS.\n\n5 Discussion\n\nWe have reproduced a variety of \ufb01ndings related to CBS, and make two main predictions: First,\ninterfering with cortical homeostatic mechanisms after the loss of vision should delay or prevent\nthe development of hallucinations. Second, we suggest that acetylcholine could not only in\ufb02uence\nthe balance of thalamic and intracortical inputs [24], but also the balance in between bottom-up and\ntop-down at various stages of the cortical hierarchy. In CBS in particular, lack of acetylcholine at\ncortical sites should correlate with the emergence of hallucinations.\nNeurological pathologies other than CBS have been studied before in neural networks [11]. In [18]\nschizophrenia is modeled with an approach akin to ours, with hallucinatory memories surfacing in\na Hop\ufb01eld net due to homeostatic mechanisms that compensate for input degradation. However,\nthere the \u2018memories\u2019, supposedly residing in prefrontal cortex, are accounted for much more ab-\nstractly, consisting of hard-coded random patterns. In our model, these unspeci\ufb01ed memories can\nbe understood as learned latent representations in a hierarchical generative model of visual input.\nThe explicit image-based representations made it possible to investigate localized degradation of\nvisual input, and the hierarchical nature of the DBM allowed us to examine lesions and suppression\nwithin the cortex, and to model acetylcholine as mediating the feed-forward/feed-back balance of\ninformation \ufb02ow. Moreover, the present work needs to be seen not just in the context of models\nof speci\ufb01cally mental dysfunction, but also in the context of models attempting to capture general\nprinciples of learning and perception in the visual cortex. Here, generative models of unsupervised\nlearning are promising as they can naturally account for the formation of internal imagery in health\nand disease. We emphasize that the key aspect of a model of visual hallucinations is not that it\ngenerates images, but that it spontaneously generates rich internal representations of images.\nWe only have used toy data. As current machine learning work sees DBMs applied to more and more\ncomplex problems, more powerful demonstrations of complex hallucinations should be possible in\nthe future. Also, other hallucinatory pathologies could be explored, such as schizophrenia. One neu-\nrological abnormality implicated in the latter is a potential disconnection of different cortical regions\n[26]. In the DBM, this could be modeled by decoupling different parts of the architecture, and incor-\nporating other sensory hierarchies to account for the fact that visual hallucinations in schizophrenia\ntend to come with auditory hallucinations, suggesting system wide interactions.\nAnother interesting issue is the nature of (non-hallucinatory) mental imagery. Why is the perceptual\nquality of mental imagery so less salient than that of vivid hallucinations? We suggest that for men-\ntal imagery, representations are merely realized in higher areas that code for objects more abstractly,\nwhereas for vivid hallucinations they are realized throughout the whole system [13], and hence are\nricher in information content. In the cortex, mechanisms such as in-built translation invariance (com-\nplex cell pooling) likely lead to some information not being represented in higher areas, something\nnot explicitly accounted for in our model. In that context it is thus very interesting to see recent\nattempts [7, 27] at implementing biologically related mechanisms (such as lateral interactions) in\nDBM-like models that could invert this information loss when generating images: The idea is that\nhigher layers only seed images in an approximate fashion, and lower areas sort out the details, by\naligning edges and so forth. Then, lower areas really would be needed to realize all information\nentailed in rich perception, thus explaining the perceptual difference in between high level mental\nimagery and system wide vivid visual hallucinations.\n\nAcknowledgments\n\nWe would like to thank Nicolas Heess for helpful comments, Geoff Hinton for input on the mecha-\nnism underlying the ACh model (cf. [28]), and the EPSRC, MRC and BBSRC for funding.\n\n8\n\n\fReferences\n[1] Manford, M. and Andermann, F. (1998) Complex visual hallucinations. clinical and neurobiological in-\n\nsights. Brain, 121, 1819\u20131840.\n\n[2] Collerton, D., Perry, E., and McKeith, I. (2005) Why people see things that are not there: A novel per-\nception and attention de\ufb01cit model for recurrent complex visual hallucinations. Behavioral and Brain\nSciences, 28, 737\u2013757.\n\n[3] Schultz, G. and Melzack, R. (1991) The Charles Bonnet Syndrome: \u2019phantom visual images\u2019. Perception,\n\n20, 809\u2013825, PMID: 1816537.\n\n[4] Teunisse, R. J., Zitman, F. G., Cruysberg, J. R. M., Hoefnagels, W. H. L., and Verbeek, A. L. M. (1996)\nVisual hallucinations in psychologically normal people: Charles Bonnet\u2019s Syndrome. The Lancet, 347,\n794\u2013797.\n\n[5] Menon, G. J., Rahman, I., Menon, S. J., and Dutton, G. N. (2003) Complex visual hallucinations in the\nvisually impaired: the Charles Bonnet Syndrome. Survey of Ophthalmology, 48, 58\u201372, PMID: 12559327.\n[6] Rao, R. P. and Ballard, D. H. (1999) Predictive coding in the visual cortex: a functional interpretation of\n\nsome extra-classical receptive-\ufb01eld effects. Nature Neuroscience, 2, 79\u201387, PMID: 10195184.\n\n[7] Hinton, G. E. (2010) Learning to represent visual input. Philosophical Transactions of the Royal Society\n\nB: Biological Sciences, 365, 177\u2013184.\n\n[8] Friston, K. J. (2005) Hallucinations and perceptual inference. Behavioral and Brain Sciences, 28, 764\u2013\n\n766.\n\n[9] Corlett, P., Frith, C., and Fletcher, P. (2009) From drugs to deprivation: a Bayesian framework for under-\n\nstanding models of psychosis. Psychopharmacology, 206, 515\u2013530.\n\n[10] Salakhutdinov, R. and Hinton, G. (2009) Deep Boltzmann machines. Proceedings of the 12th Interna-\n\ntional Conference on Arti\ufb01cial Intelligence and Statistics (AISTATS), vol. 5, pp. 448\u2013455.\n\n[11] Finkel, L. H. (2000) Neuroengineering models of brain disease. Annual Review of Biomedical Engineer-\n\ning, 2, 577\u2013606.\n\n[12] Lee, H., Ekanadham, C., and Ng, A. Y. (2008) Sparse deep belief net model for visual area V2. Advances\n\nin Neural Information Processing Systems 20.\n\n[13] Pollen, D. A. (1999) On the neural correlates of visual perception. Cerebral Cortex, 9, 4\u201319.\n[14] Turrigiano, G. G. and Nelson, S. B. (2000) Hebb and homeostasis in neuronal plasticity. Current Opinion\n\nin Neurobiology, 10, 358\u2013364.\n\n[15] Houweling, A. R., Bazhenov, M., Timofeev, I., Steriade, M., and Sejnowski, T. J. (2005) Homeostatic\nsynaptic plasticity can explain post-traumatic epileptogenesis in chronically isolated neocortex. Cereb.\nCortex, 15, 834\u2013845.\n\n[16] ffytche, D. H. and Howard, R. J. (1999) The perceptual consequences of visual loss: \u2018positive\u2019 pathologies\n\nof vision. Brain, 122, 1247\u20131260.\n\n[17] Hinton, G. E. (2007) Boltzmann machine. Scholarpedia, 2, 1668.\n[18] Ruppin, E., Reggia, J. A., and Horn, D. (1996) Pathogenesis of schizophrenic delusions and hallucina-\n\ntions: a neural model. Schizophrenia Bulletin, 22, 105\u2013123, PMID: 8685653.\n\n[19] Hinton, G. E. and Salakhutdinov, R. R. (2006) Reducing the dimensionality of data with neural networks.\n\nScience, 313, 504\u2013507.\n\n[20] Tsotsos, J. K., Rodriguez-Sanchez, A. J., Rothenstein, A. L., and Simine, E. (2008) The different stages\n\nof visual recognition need different attentional binding strategies. Brain Research, 1225, 119\u2013132.\n\n[21] Nair, V. and Hinton, G. E. (2009) 3D object recognition with deep belief nets. Advances in Neural Infor-\n\nmation Processing Systems 22.\n\n[22] Merabet, L. B., Kobayashi, M., Barton, J., and Pascual-Leone, A. (2003) Suppression of complex visual\nhallucinatory experiences by occipital transcranial magnetic stimulation: A case report. Neurocase: The\nNeural Basis of Cognition, 9, 436.\n\n[23] Grossberg, S. (2000) How hallucinations may arise from brain mechanisms of learning, attention, and\n\nvolition. Journal of the International Neuropsychological Society, 6, 583\u2013592.\n\n[24] Sarter, M., Hasselmo, M. E., Bruno, J. P., and Givens, B. (2005) Unraveling the attentional functions of\ncortical cholinergic inputs: interactions between signal-driven and cognitive modulation of signal detec-\ntion. Brain Research. Brain Research Reviews, 48, 98\u2013111, PMID: 15708630.\n\n[25] Yu, A. J. and Dayan, P. (2002) Acetylcholine in cortical inference. Neural Networks: The Of\ufb01cial Journal\n\nof the International Neural Network Society, 15, 719\u2013730, PMID: 12371522.\n\n[26] Ellison-Wright, I. and Bullmore, E. (2009) Meta-analysis of diffusion tensor imaging studies in\n\nschizophrenia. Schizophrenia Research, 108, 3\u201310, PMID: 19128945.\n\n[27] Osindero, S. and Hinton, G. (2008) Modeling image patches with a directed hierarchy of Markov random\n\n\ufb01elds. Advances in Neural Information Processing Systems, 20.\n\n[28] Hinton, G. E. (2006) Unsupervised learning for perception. NSERC Discorvery Grant Proposal, available\n\nfrom the author.\n\n9\n\n\f", "award": [], "sourceid": 932, "authors": [{"given_name": "Peggy", "family_name": "Series", "institution": null}, {"given_name": "David", "family_name": "Reichert", "institution": null}, {"given_name": "Amos", "family_name": "Storkey", "institution": null}]}