{"title": "A Log-Domain Implementation of the Diffusion Network in Very Large Scale Integration", "book": "Advances in Neural Information Processing Systems", "page_first": 2487, "page_last": 2495, "abstract": "The Diffusion Network(DN) is a stochastic recurrent network which has been shown capable of modeling the distributions of continuous-valued, continuous-time paths. However, the dynamics of the DN are governed by stochastic differential equations, making the DN unfavourable for simulation in a digital computer. This paper presents the implementation of the DN in analogue Very Large Scale Integration, enabling the DN to be simulated in real time. Moreover, the log-domain representation is applied to the DN, allowing the supply voltage and thus the power consumption to be reduced without limiting the dynamic ranges for diffusion processes. A VLSI chip containing a DN with two stochastic units has been designed and fabricated. The design of component circuits will be described, so will the simulation of the full system be presented. The simulation results demonstrate that the DN in VLSI is able to regenerate various types of continuous paths in real-time.", "full_text": "A Log-Domain Implementation of the Diffusion\n\nNetwork in Very Large Scale Integration\n\nYi-Da Wu, Shi-Jie Lin, and Hsin Chen\nDepartment of Electrical Engineering\n\nNational Tsing Hua University\n\nHsinchu, Taiwan 30013\n\n{ydwu;hchen}@ee.nthu.edu.tw\n\nAbstract\n\nThe Diffusion Network(DN) is a stochastic recurrent network which has been\nshown capable of modeling the distributions of continuous-valued, continuous-\ntime paths. However, the dynamics of the DN are governed by stochastic differ-\nential equations, making the DN unfavourable for simulation in a digital computer.\nThis paper presents the implementation of the DN in analogue Very Large Scale\nIntegration, enabling the DN to be simulated in real time. Moreover, the log-\ndomain representation is applied to the DN, allowing the supply voltage and thus\nthe power consumption to be reduced without limiting the dynamic ranges for dif-\nfusion processes. A VLSI chip containing a DN with two stochastic units has been\ndesigned and fabricated. The design of component circuits will be described, so\nwill the simulation of the full system be presented. The simulation results demon-\nstrate that the DN in VLSI is able to regenerate various types of continuous paths\nin real-time.\n\n1\n\nIntroduction\n\nIn many implantable biomedical microsystems [1, 2], an embedded system capable of recognis-\ning high-dimensional, time-varying signals have been demanded. For example, recognising multi-\nchannel neural activity on-line is important for implantable brain-machine interfaces to avoid trans-\nmitting all data wirelessly, or to control prosthetic devices and to deliver bio-feedbacks in real-\ntime [3].\nThe Diffusion Network (DN) proposed by Movellan is a stochastic recurrent network whose stochas-\ntic dynamics can be trained to model the probability distributions of continuous-time paths by the\nMonte-Carlo Expectation-Maximisation (EM) algorithm [4, 5]. As stochasticity is useful for gener-\nalising the natural variability in data [6, 7], the DN is further shown suitable for recognising noisy,\ncontinuous-time biomedical data [8]. However, the stochastic dynamics of the DN is de\ufb01ned by a\nset of continuous-time, stochastic differential equations (SDEs). The speed of simulating stochas-\ntic differential equations in a digital computer is inherently limited by the serial processing and\nnumerical iterations of the computer. Translating the DN into analogue circuits is thus of great in-\nterests for simulating the DN in real time by exploiting the natural, differential current-voltage (I-V)\nrelationship of capacitors [9].\nThis paper presents the implementation of the DN in analogue Very Large Scale Integration (VLSI).\nTo minimise the power consumption, the power supply voltage is only 1.5V, and most transistors are\noperated in subthreshold regions. As the reduced supply voltage limits directly the dynamic range\navailable for voltages across capacitors, the log-domain representation proposed in [10] is applied\nto the DN, allowing diffusion processes to be simulated in a limited voltage ranges. After a brief\n\n1\n\n\fintroduction to the DN, the following sections will derive the log-domain representation of the DN\nand describe its corresponding implementation in analogue VLSI.\n\nxj\n\n\u03c9ji\n\n\u03c9jj\n\n\u03c9ij\n\nxi\n\n\u03c9ii\n\nxk\n\n\u03c9kk\n\nFigure 1: The architecture of\na Diffusion Network with one\nvisible and two hidden units\n\n2 The Diffusion Network\n\n\u00b7 dB\n\ndt\n\n\u03c3\n\u03baj\n\n\u03bej + \u03a3\u03c9ij\u03d5i\n\n\u03bej\n\nVXj\n\nxj\n\n\u03d5(xi)\n\n\u03c9ij\n\n\u03c9jj\n\n\u03c1jxof f\n\n\u03c1jIs\n\nCXj\n\nIs\n\nxof f\n\n\u03d5(xj)\n\nFigure 2: The block diagram of a DN unit in VLSI\n\nAs shown in Fig. 1, the DN comprises n continuous-time, continuous-valued stochastic units with\nfully recurrent connections. The state of the jth unit at time t, xj(t), is governed by\n\n(cid:0)xj(t)(cid:1) + \u03c3 \u00b7 dB(t)\n\ndt\n\ndxj(t)\n\ndt\n\n= \u00b5j\n\nwhere \u00b5j(t) is a deterministic drift term given in (2), \u03c3 a constant, and dB(t) the Brownian motion.\nThe Brownian motion introduces the stochasticity, enriching greatly the representational capability\nof the DN [5].\n\n(cid:0)xj(t)(cid:1) = \u03baj \u00b7\n\n\u00b5j\n\n(cid:20)\n\n\u2212\u03c1jxj(t) + \u03bej +\n\n\u03c9ij de\ufb01nes the connection weight from unit i to unit j. \u03ba\u22121\nrepresent the input capacitance\nand transmembrane resistance, respectively, of the jth unit. \u03bej is the input bias, and \u03d5 is the sigmoid\nfunction given as\n\nand \u03c1\u22121\n\nj\n\n\u03c9ij \u00b7 \u03d5(cid:0)xi(t)(cid:1)(cid:21)\n\nnX\n\ni=1\n\nj\n\n(cid:16) a\n\n(cid:17)\n\n(1)\n\n(2)\n\n(3)\n\n\u03d5(xj; a) = \u22121 +\n\n2\n\n1 + e\u2212axj\n\n= tanh\n\n2 xj\n\nwhere a adapts the slope of the sigmoid function. As shown in Fig. 1, the DN contains both visi-\nble(white) and hidden(grey) stochastic units. The learning of the DN aims to regenerate at visible\nunits the probability distribution of a speci\ufb01c set of continuous paths. The number of visible units\nthus equals the dimension of the data to be modeled, while the minimum number of hidden units\nrequired for modeling data satisfactorily is identi\ufb01ed by experimental trials. During training, visible\nunits are \u201cclamped\u201d to the dynamics of the training dataset, and the dynamics of hidden units are\nMonte-Carlo sampled for estimating optimal parameters (\u03c9ij, \u03baj, \u03c1j, \u03bej) that maximise the expec-\ntation of training data [5]. After training, all units are given initial values at t = 0 only to sample\nthe dynamics modeled by the DN. The similarity between the dynamics of visible units and those of\ntraining data indicate how well the DN models the data.\n\n2.1 Log-domain translation\n\nTo maximise the dynamic ranges for diffusion processes in VLSI, the stochastic state xj(t) is rep-\nresented as a current and then logarithmically-compressed into a voltage VXj in VLSI [11]. The\nlogarithmic compression allows xj(t) to change over three decades within a limited voltage range\nfor VXj. The voltage representation VXj further facilitates the exploitation of the nature, differential\n(I-V) relationship of a capacitor to simulate SDEs in real-time and in parallel.\n\n2\n\nEXPEXPEXPEXP\fThe logarithmic relationship between xj(t) and VXj can be realised by the exponential I-V char-\nacteristics of a MOS transistor in subthreshold operation [12]. To keep xj(t) a non-negative value\n(current) in VLSI, an offset xof f is added to xj(t), resulting in the following relationship between\nxj(t) and VXj.\n\nxj + xof f \u2261 IS \u00b7 e\u03b1VXj , dxj = \u03b1IS \u00b7 e\u03b1VXj \u00b7 dVXj\n\n(4)\n\nwhere Is and \u03b1 are process-dependent constants extractable from simulated I-V curves of transistors.\nSubstituting Eq. (4) into Eq. (1) then translates the diffusion process in Eq. (1) into the following\nequation.\n\n(cid:20)\n\nnX\n\ni=1\n\n(cid:21)\n\nCXj \u00b7 dVXj\ndt\n\n=\n\n\u03bej +\n\n\u03c9ij\u03d5(xi)\n\n\u00b7 e\u2212\u03b1VXj + \u03c3\n\u03baj\n\ndBj(t)\n\ndt\n\n\u00b7 e\u2212\u03b1VXj + \u03c1jxof f \u00b7 e\u2212\u03b1VXj \u2212 \u03c1jIS\n(5)\n\nwhere CXj equals \u03b1/\u03baj. Fig. 2 illustrates the block diagram for implementing Eq. (5) in VLSI.\nCXj is a capacitor and VXj the voltage across the capacitor. Each term on the right hand side of\nEq. (5) then corresponds to a current \ufb02owing into CXj. Let (VP \u2212 VN ) and IV AR represent the\ndifferential input voltage and the input current of an EXP-element, respectively. Each EXP-element\nin Fig. 2 produces an output current of Iout = IV AR \u00b7 e\u03b1(VP \u2212VN ). Therefore, the EXP-elements\nimplement the \ufb01rst three terms multiplied with e\u2212\u03b1VXj in accordance with Eq. (5). The last term,\n\u03c1jIS, is a constant and is thus implemented by a constant current source. Finally, the sigmoid circuit\ni=1 \u03c9ij \u00b7 \u03d5(xi).\n\ntransforms xj into \u03d5(xj) and the multipliers output a total current proportional toPn\n\nFigure 3: The stochastic dynamics (gray lines)\nregenerated by the DN trained on the bifurcating\ncurves (black lines).\n\nFigure 4: The stochastic dynamics (gray lines)\nregenerated by the DN trained on the sinusoidal\ncurve (the black line).\n\nFigure 5: The stochastic dynamics (gray lines) re-\ngenerated by the DN trained on the QRS segments of\nelectrocardiograms (black lines).\n\nFigure 6: The stochastic dynamics (gray\nlines) regenerated by the DN trained on the\nhandwritten \u03c1 (the black line).\n\n2.2 Adapting \u03c1j instead of \u03baj\n\nThe DN has been shown capable of modeling various distributions of continuous paths by adapting\nwij, \u03bej, and \u03baj in [5]. An adaptable \u03baj corresponds to an adaptable CXj, but a tunable capacitor with\na wide linear range is not easy to implement in VLSI. As Eq. (2) indicates that \u03c1j is complementary\n\n3\n\n 3 4 5 6 7 0 100 200 300 400 500Time samples 3 4 5 6 7 0 100 200 300 400 500 600 700 800 900 1000Time samples 2.5 3.5 4.5 5.5 6.5 7.5 0 20 40 60 80Time samples 4 5 6 7 8 9 10 3.5 4.5 5.5 6.5 7.5 8.5Unit 2Unit 1\fto \u03baj in determining the \u201ctime constant\u201d of the dynamics of the unit j, the possibility of adapting \u03c1j\ninstead of \u03baj is investigated by Matlab simulation.\nWith \u03baj = 1, the DN was trained to model different data by adapting \u03c9ij, \u03bej, and \u03c1j for 100 epochs.\nA DN with one visible and one hidden units was proved capable of regenerating the dynamics of\nbifurcating curves (Fig. 3), sinusoidal waves (Fig. 4), and electrocardiograms (Fig. 5). Moreover, a\nDN with only two visible units was able to regenerate the handwritten \u03c1 satisfactorily, as illustrated\nin Fig. 6. The promising results supported the suggestion that adapting \u03c1j instead of \u03baj also allowed\nthe DN to model different data. As a variable \u03c1j simply corresponded to a tunable current source\n\u03c1jIS in Fig. 2, the VLSI implementation was greatly simpli\ufb01ed.\n\n2.3 Parameter mappings\n\nTable 1 summarises the parameter mappings between the numerical simulation and the VLSI im-\nplementation. All variables except for VXj in Fig. 2 are represented as currents in VLSI. The unit\ncurrents (Iunit) of xj, \u03c9ij, and \u03bej are de\ufb01ned as 10 nA to match the current scales of transistors\nin subthreshold operation, as well as to reduce the power consumption. Moreover, extensive simu-\nlations indicate that the dynamic ranges required for modeling various data are [\u22123, 5] for xj and\n[\u221230, 30] for \u03c9ij. With xof f = 5 in Eq. (4), i.e. xof f = 50nA in VLSI, VXj ranges from 773\nto 827 mV. While the diffusion process in Eq. (1) is iterated with \u2206t = 0.05 in numerical simula-\ntion, \u2206t = 0.05 is set to be 5 \u00b5s in VLSI, corresponding to a reasonable sampling rate (200kHz)\nat which most instruments can sample multiple channels(units) simultaneously. Finally, the unit\ncapacitance for 1/\u03baj is calculated as Cunit = Iunit \u00b7 \u2206tunit/VXj,unit, equaling 1 pF and resulting\nin CXj = \u03b1 \u00b7 Cunit = 30 pF.\n\nTable 1: Parameter mappings between numerical simulation and VLSI implementation\n\nparameter\n\nxj\nxof f\nVXj\n\u03c9, \u03be\n\u03d5(xj)\nCXj\n\u2206t\n\u03c1\n\nnumeric\n-3\u223c5\n5\n\n-30\u223c30\n-1\u223c1\n\n0.773\u223c0.827\n\n\u03b1/\u03baj = 30\n\n0.05\n0.5\u223c2\n\ncircuit\n\n50 nA\n\n-30\u223c50 nA\n773\u223c827 mV\n-300\u223c300 nA\n-400\u223c400 nA\n\n30 pF\n5 \u00b5s\n0.5\u223c2\n\ncomment\n\nIunit = 10 nA\n\noffset term in Eq. (4)\n\nVXj,unit = 1 V\nIunit = 10 nA\n\nactivation function\n\nCunit = 1 pF\ntunit = 0.1 ms\n\n3 Circuit implementation\n\nA DN with two stochastic units have been designed with the CMOS 0.18 \u00b5m technology provided by\nthe Taiwan Semiconductor Manufacturing Company (TSMC). The following subsections introduce\nthe design of each component circuit.\n\n3.1 The EXP element\n\nFig. 7(b) shows the schematics of the EXP element. With M1 and M2 operated in the subthreshold\nregion, the output current is given as\n\nIout = IB \u00b7 exp\n\n(VP \u2212 VN )\n\n(6)\n\n(cid:16) 1\n\nnUT\n\n(cid:17)\n\nwhere UT denotes the thermal voltage and n the subthreshold slope factor. Comparing Eq. (6) with\nEq. (4) reveals that \u03b1 = 1/nUT . As the drain current (Id) of a transistor in subthreshold operation\nis exponentially proportional to its gate-to-source voltage (VGS) as Id \u221d eVGS /nUT , \u03b1 = 1/nUT is\nextracted to be 30 by plotting log(Id) versus VGS in SPICE.\nTransistors M3-M5 form an active biasing circuit that sinks IB + Iout. By adjusting the gate voltage\nof M3 through the negative feedback, Iout is allowed to change over several decades. In addition,\n\n4\n\n\fn actually depends on the gate voltage and introduces variability to \u03b1 [13]. To prevent the variable\n\u03b1 from introducing simulation errors, all EXP elements of the DN unit are biased with a constant\nIB = 100 nA. As shown by Fig. 7(a), Iout of each element is then re-scaled by the one-quadrant\nout = Iout \u00d7 IV AR/IB,\ncurrent multiplier basing on translinear loops (Fig. 7(c)) [13] to produce I0\nwhere IV AR represents the current input to each element in Fig. 2 (e.g.\u03a3\u03c9\u03d5 or \u03c1xof f ).\n\nIB\n\nIOU T\n\nT\nU\nO\nI\n\nB\nI\n\nR\nA\nV\nI\n\nT\nU\n0O\nI\n\nVP\n\nVN\n\nIOU T\n\nIB\n\nI0\n\nOU T\n\nIB\n\nIV AR\n\n(a)\n\n(b)\n\n(c)\n\nFigure 7: The circuit diagram of the EXP element.\n\n3.2 Current multipliers\n\nFour-quadrant multipliers basing on translinear loops [13] are employed to calculate \u03a3\u03c9ij\u03d5(xi) in\nEq. (5). Both \u03c9ij and \u03d5(xi) are represented by differential currents as\n\n\u03c9ij = I\u03c9+ \u2212 I\u03c9\u2212 , \u03d5(xi) = I\u03d5+ \u2212 I\u03d5\u2212\n\n(7)\n\nLet the differential current (IZ+ \u2212 IZ\u2212) represents the multiplier\u2019s output and IU represent a unit\ncurrent. Eq. (8) indicates that the four-quadrant multiplication can be composed of four one-quadrant\nmultipliers in Fig. 7(c), as illustrated in Fig. 8.\n\nIZ+ \u00b7 IU \u2212 IZ\u2212 \u00b7 IU = (I\u03c9+ \u00b7 I\u03d5+ + I\u03c9\u2212 \u00b7 I\u03d5\u2212) \u2212 (I\u03c9+ \u00b7 I\u03d5\u2212 + I\u03c9\u2212 \u00b7 I\u03d5+)\n\n(8)\n\nFig. 9 shows the simulation result of the four-quadrant multiplier, exhibiting satisfactory linearity\nover the dynamic ranges required in Table 1.\n\n+\n\u03d5\nI\n\n\u2212\n\u03d5\nI\n\n\u2212\n\u03d5\nI\n\n+\n\u03d5\nI\n\nIU\n\nIU\n\n\u2212\n\u03c9\nI\n\n+\n\u03c9\nI\n\nIU\n\n\u2212\n\u03c9\nI\n\nIZ\u2212\n\nIZ+\n\nIU\n\n+\n\u03c9\nI\n\nFigure 8: The four-quadrant current multiplier\n\n5\n\nEXPM1M2VNVPM3M5VSM4VbiasnVbiasnM6M5M1M4M3M2M7Vref\fA\nn\n\nn\ni\n\n)\n\u2212\nZ\nI\n\u2212\n+\nZ\nI\n(\n\n\u03d5i = \u2212400nA\n\u03d5i = \u2212300nA\n\u03d5i = \u2212200nA\n\u03d5i = \u2212100nA\n\n\u03d5i = 0\n\u03d5i = 100nA\n\u03d5i = 200nA\n\u03d5i = 300nA\n\u03d5i = 400nA\n\n(I\u03c9+ \u2212 I\u03c9\u2212) in nA\n\nFigure 9: The simulation results of the four-\nquadrant current multiplier\n\nFigure 10: The simulation result of the sigmoid\ncircuit with different Va\n\n3.3 Sigmoid function \u03d5(\u00b7)\n\nFig. 11 shows the block diagram for implementing the sigmoid function in Eq. (3). The current\nIXi representing xi is \ufb01rstly converted into a voltage Vi by the the operational ampli\ufb01er(OPA)\nwith a voltage-controlled active resistor (VCR) proposed in [14]. Vi is then sent to an operational\ntransconductance ampli\ufb01er(OTA) in subthreshold operation, producing an output current of\n\n(Vi \u2212 Vref )\n\nIs = IB tanh\n\n(9)\nSince Vi \u2212 Vref = Ri \u00b7 Ixi, with Ri representing the resistance of the VCR, the voltage Va adapts\nRi and thus the slope of the sigmoid function. Finally, the 2nd generation current conveyor (CCII)\nin Fig. 12 [15] converts the current Is into a pair of differential currents (IOU T N , IOU T P ) ranging\nbetween \u2212400 nA and +400 nA. The differential currents are then duplicated for the inputs of\nfour-quadrant multipliers of all DN units.\n\n(cid:16) 1\n\n2nUT\n\n(cid:17)\n\nIXi\n\nIOU T P\nIOU T N\n\nFigure 11: The block diagram of the sigmoid circuit.\n\n3.4 Capacitor ampli\ufb01cation\nAs CXi = 30 pF requires consider-\nable chip area, CXi is implemented\nby the circuit in Fig. 13, utilising the\nMiller effect to amplify the capaci-\ntance. Let A denote the gain of the\nampli\ufb01er. The effective capacitance\nbetween X and Y is (1 + A) \u00b7 CX.\nFig. 13 also shows the schematics of\nthe ampli\ufb01er whose gain is designed\nto be 2. As a result, CX = 10 pF is\nsuf\ufb01cient for providing an effective\nCXi of 30 pF.\n\nVBIAS\n\nX\n\nY\n\nCX\n\nVREF\n\nX\n\nY\n\nCEQ = CX(1 + A)\n\nFigure 13: The circuit diagram of the capacitor ampli-\n\ufb01ed by the Miller effect.\n\n6\n\n\u2212200\u2212100 0 100 200\u2212400 \u2212200 0 200400-500-400-300-200-100 0 100 200 300 400 500-600-400-200 0 200 400 600Output current in nAInput current in nAgain=0.8gain=1.0gain=3.0gain=5.0CCIIOPAVCRVrefVaVrefVrefOTA4/4x14/4x16\u2212AM2M1\fVX\n\nVREF\n\nIOU T N\n\nIP\n\nVY\n\nIN\n\nIOU T P\n\nFigure 12: The circuit diagram of the single-to-differential current conveyor\n\n1P6M 0.18 \u00b5m CMOS\n\n345 \u00b5Watts\n\n1.368\u00d71.368mm2\n\nFigure 14: The chip layout and its speci\ufb01cation.\n\nA\n\u00b5\nn\ni\n\n1\nX\nI\n\nA\n\u00b5\nn\ni\n\n1\nX\nI\n\nFigure 15: The sinusoidal dynamics regenerated\nby the DN chip in post-layout simulation (10 tri-\nals).\n\nFigure 16: The electrocardiogram dynamics re-\ngenerated by the DN chip in post-layout simu-\nlation (10 trials).\n\n7\n\nVbiasnVbiaspIsigOPA0.3V0.3V1.2V1.2V21.5 VoltsChip AreaNum. of UnitsPower ConsumptionPower SupplyTechnology1D/2D continuous paths(including pads)CapabilityMax. Bandwidth1.6 kHz 30 40 50 0 0.5 1 1.5 2Time in ms 30 35 40 45 50 55 0 0.05 0.1 0.15 0.2 0.25 0.3Time in ms\fA\n\u00b5\nn\ni\n\n1\nX\nI\n\nA\n\u00b5\nn\ni\n\n2\nX\nI\n\nIX1 in \u00b5A\n\nFigure 17: The bifurcating dynamics regenerated by\nthe DN chip in post-layout simulation (8 trials).\n\nFigure 18: The handwritten \u03c1 regenerated\nby the DN chip in post-layout simulation\n(10 trials).\n\n4 The Diffusion Network in VLSI\n\nFig. 14 shows the chip layout of the log-domain implementation of the DN with two stochastic\nunits, so is the speci\ufb01cation shown. The area of the core circuit and the capacitors are 0.306 mm2\nand 0.384 mm2, respectively. The total power consumption is merely 345 \u00b5W, by the merit of low\nsupply voltage (1.5V) and subthreshold operation. The chip has been taped out for fabrication with\nthe CMOS 0.18 \u00b5m Technology by the TSMC. The post-layout simulations are shown in Fig. 15\u221218\nand described as follows.\nWith one unit functioning as a visible unit and the other as a hidden unit, the parameters of the DN\nwas programmed to regenerate the one-dimensional paths in Sec. 2.2. The noise current \u03c3\ndt was\nsimulated by a piecewise-linear current source with random amplitudes in the SPICE. As shown\nby Fig. 15-17, the visible unit was capable of regenerating the sinusoidal waves, the electrocardio-\ngrams, and the bifurcating curves with negligible differences from Fig. 3-5. Moreover, as both units\nfunctioned as visible units, the DN was capable of regenerating the handwritten \u03c1 as Fig. 18. These\npromising results demonstrate the capability of the DN chip to model the distributions of different\ncontinuous paths reliably and power-ef\ufb01ciently. After chip is fabricated in August, the chip will be\ntested and the measurement results will be presented in the conference.\n\n\u03ba \u00b7 dB\n\n5 Conclusion\n\nThe log-domain representation of the Diffusion Network has been derived and translated into ana-\nlogue VLSI circuits. Based on well-de\ufb01ned parameter mappings, the DN chip is proved capable of\nregenerating various types of continuous paths, and the log-domain representation allows the dif-\nfusion processes to be simulated in real-time and within a limited dynamic range. In other words,\nanalogue VLSI circuits are proved useful for solving (simulating) multiple SDEs in real-time and in\na power-ef\ufb01cient manner. After verifying the chip functionality, a DN chip with a scalable number\nof units will be further developed for recognising multi-channel, time-varying biomedical signals in\nimplantable microsystems.\n\nAcknowledgments\n\nThe authors thank National Chip Implementation Center (CIC) for fabrication services, and Mr.\nC.-M. Lai and S.-C. Sun for helpful discussions.\n\n8\n\n 20 30 40 50 60 0 0.5 1 1.5 2 2.5Time in ms 10 20 30 40 50 60 70 10 15 20 25 30 35 40 45 50 55\fReferences\n[1] G. Iddan, G. Meron, A. Glukhovsky, and P. Swain, \u201cWireless capsule endoscopy,\u201d Nature, vol.\n\n405, no. 6785, p. 417, July 2000.\n\n[2] T. W. Berger, M. Baudry, J.-S. L. Roberta Diaz Brinton, V. Z. Marmarelis, A. Y. Park, B. J.\nSheu, and A. R. Tanguay, JR., \u201cBrain-implantable biomimetic electronics as the next era in\nneural prosthetics,\u201d Proc. IEEE, vol. 89, no. 7, pp. 993\u20131012, July 2001.\n\n[3] M. A. Lebedev and M. A. L. Nicolelis, \u201cBrain-machine interfaces: past, present and future,\u201d\n\nTrends in Neuroscience, vol. 29, no. 9, pp. 536\u2013546, 2006.\n\n[4] J. R. Movellan, \u201cA learning theorem for networks at detailed stochastic equilibrium,\u201d Neural\n\nComputation, vol. 10, pp. 1157\u20131178, July 1998.\n\n[5] J. R. Movellan, P. Mineiro, and R. J.Williams, \u201cA Monte Carlo EM approach for partially\nobservable diffusion processes: Theory and applications to neural networks,\u201d Neural Compu-\ntation, vol. 14, pp. 1507\u20131544, July 2002.\n\n[6] H. Chen and A. F. Murray, \u201cA continuous restricted Boltzmann machine with an implementable\ntraining algorithm,\u201d IEE Proc. of Vision, Image and Signal Processing, vol. 150, no. 3, pp.\n153\u2013158, 2003.\n\n[7] D. F. Specht, \u201cProbabilistic neural networks,\u201d Neural Networks, vol. 3, no. 1, pp. 109\u2013118,\n\n1990.\n\n[8] Y. S. Hsu, T. J. Chiu, and H. Chen, \u201cReal-time recognition of continuous-time biomedical sig-\nnals using the diffusion network,\u201d in Proc. of the Int. Joint Conf. on Neural Networks (IJCNN),\n2008, pp. 2628\u20132633.\n\n[9] L. O. Chua, T. Roska, T. Kozek, and A. Zarandy, \u201cCNN universal chips crank up the computing\n\npower,\u201d IEEE Circuits and Devices Mag., vol. 12, no. 4, pp. 18\u201328, July 1996.\n\n[10] T. Serrano-Gotarredona and B. Linares-Barranco, \u201cLog-domain implementation of complex\ndynamics reaction-diffusion neural networks,\u201d IEEE Trans. Neural Networks, vol. 14, pp.\n1337\u20131355, Sept. 2003.\n\n[11] D. R. Frey, \u201cExponential state space \ufb01lters: A generic current mode design strategy,\u201d IEEE\n\nTrans. Circuits Syst. I, vol. 43, pp. 34\u201342, Jan. 1996.\n\n[12] E. Vittoz and J. Fellrath, \u201cCMOS analog integrated circuits based on weak inversion operation,\u201d\n\nIEEE J. Solid-State Circuits, vol. 12, pp. 224\u2013231, June 1977.\n\n[13] S.-C. Liu, J. Kramer, G. Indiveri, T. Delbr\u00a8uck, and R. Douglas, Analog VLSI: Circuits and\n\nPrinciples. The MIT Press, 2002.\n\n[14] M. Banu and Y. Tsividis, \u201cFloating voltage-controlled resistors in CMOS technology,\u201d Elec-\n\ntronics Letters, vol. 18, no. 15, pp. 678\u2013679, July 1982.\n\n[15] C. Toumazou, F. J. Lidgey, and D. G. Haigh, Analogue IC Design: The Current-Mode Ap-\n\nproach. Peter Peregrinus Ltd, 1990.\n\n9\n\n\f", "award": [], "sourceid": 538, "authors": [{"given_name": "Yi-da", "family_name": "Wu", "institution": null}, {"given_name": "Shi-jie", "family_name": "Lin", "institution": null}, {"given_name": "Hsin", "family_name": "Chen", "institution": null}]}