{"title": "Multi-effect Decompositions for Financial Data Modeling", "book": "Advances in Neural Information Processing Systems", "page_first": 995, "page_last": 1004, "abstract": null, "full_text": "Multi-effect Decompositions \nfor Financial Data Modeling \n\nLizhong Wu & John Moody \n\nOregon Graduate Institute, Computer Science Dept., \n\nPO Box 91000, Portland, OR 97291 \n\nalso at: \n\nNonlinear Prediction Systems, \n\nPO Box 681, University Station, Portland, OR 97207 \n\nAbstract \n\nHigh frequency foreign exchange data can be decomposed into three \ncomponents: the inventory effect component, the surprise infonnation \n(news) component and the regular infonnation component. The presence \nof the inventory effect and news can make analysis of trends due to the \ndiffusion of infonnation (regular information component) difficult. \nWe propose a neural-net-based, independent component analysis to sep(cid:173)\narate high frequency foreign exchange data into these three components. \nOur empirical results show that our proposed multi-effect decomposition \ncan reveal the intrinsic price behavior. \n\n1 \n\nIntroduction \n\nTick-by-tick, high frequency foreign exchange rates are extremely noisy and volatile, but \nthey are not simply pure random walks (Moody & Wu 1996). The price movements are \ncharacterized by a number of \"stylized facts\" \n,including the following two properties: \n(1) short tenn, weak oscillations on a time scale of several ticks and (2) erratic occurrence \nof turbulence lasting from minutes to tens of minutes. Property (1) is most likely caused \nby the market makers' inventory effect (O'Hara 1995), and property (2) is due to surprise \ninformation, such as news, rumors, or major economic announcements. The price changes \ndue to property (1) are referred to as the inventory effect component, and the changes due \nto property (2) are referred to as the surprise infonnation component. The price changes \ndue to other infonnation is referred to as the regular infonnation component. \n\nIThis terminology is borrowed from the financial economics literature. For additional properties \nof high frequency foreign exchange price series. see (Guilaumet, Dacorogna. Dave. Muller. Olsen & \nPictet 1994). \n\n\f996 \n\nL. Wu and 1. E. Moody \n\nDue to the inventory effect, price changes show strong negative correlations on short time \nscales (Moody & Wu 1995). Because of the surprise infonnation effect, distributions of \nprice changes are non-nonnal (Mandelbrot 1963). Since both the inventory effect and \nthe surprise information effect are short tenn and temporary, their corresponding price \ncomponents are independent of the fundamental price changes. However, their existence \nwill seriously affect data analysis and modeling (Moody & Wu 1995). Furthennore, the \nmost reliable component of price changes, for forecasting purposes, is the long tenn trend. \nThe presence of high frequency oscillations and short periods ofturbulence make it difficult \nto identify and predict the changes in such trends, if they occur. \n\nIn this paper, we propose a novel approach with the following price model: \n\nq(t) = CIPI(t) + C2P2(t) + C3P3(t) + e(t). \n\n(1) \nIn this model, q(t) is the observed price series and PI (t), P2(t) and P3(t) correspond \nrespectively to the regular infonnation component, the surprise infonnation component and \nthe inventory effect component. PI (t), P2 (t) and P3 (t) are mutually independent and may \nindividually be either iid or correlated. e(t) is process noise, and c}, C2 and C3 are scale \nconstants. Our goal is to find PI (t), p2(t) and P3( t) given q(t). \nThe outline of the paper is as follows. We describe our approach for multi-effect decompo(cid:173)\nsition in Section 2. In Section 3, we analyze the decomposed price components obtained \nfor the high frequency foreign exchange rates and characterize their stochastic properties. \nWe conclude and discuss the potential applications of our multi-effect decomposition in \nSection 4. \n\n2 Multi-effect Decomposition \n\n2.1 \n\nIndependent Source Separation \n\nThe task of decomposing the observed price quotes into a regular infonnation component, \na surprise infonnation component and an inventory effect component can be exactly fitted \ninto the framework of independent source separation. Independent source separation can \nbe described as follows: \n\nAssume that X = {Xi, i = 1, 2, ... , n} are the sensor outputs which \nare some superposition of unknown independent sources S = {Si' i = \n1, 2, ... , m }. The task of independent source separation is to find a \nmapping Y = f (X), so that Y ~ AS, where A is an m x m matrix in \nwhich each row and column contains only one non-zero element. \n\nApproaches to separate statistically-independent components in the inputs include \n\n\u2022 Blind source separation (Jutten & Herault 1991), \n\u2022 Infonnation maximization (Linsker 1989), (Bell & Sejnowski 1995), \n\u2022 Independent component analysis, (Comon 1994), (Amari, Cichocki & Yang 1996), \n\u2022 Factorial coding (Barlow 1961). \n\nAll of these approaches can be implemented by artificial neural networks. The network \narchitectures can be linear or nonlinear, multi-layer perceptrons, recurrent networks or other \ncontext sensitive networks (pearlmutter & Parra 1997). We can choose a training criterion \nto minimize the energy in the output units, to maximize the infonnation transferred in \nthe network, to reduce the redundancies between the outputs, or to use the Edgeworth \nexpansion or Gram-Charlier expansion of a probability distribution, which leads to an \nanalytic expression of the entropy in tenns of measurable higher order cumulants. \n\n\fMulti-effect Decompositions/or Financial Data Modeling \n\nr1(t) \n\nOrthogonal \n\nq(t) \n\nMulti-scale \n\nSmoothing e r2(t) \n\nIndependent \n\nComponent \n\nDecomposition \n\nr3(t) \n\nAnalysis \n\n997 \n\n~(t) \n\n~(t) \n\nFigure 1: System diagram of multi-effect decomposition for high frequency foreign ex(cid:173)\nchange rates. q(t) are original price quotes, ri(t) are the reference inputs, and Pie t) are the \ndecomposed components. \n\nFor our price decomposition problem, the non-Gaussian nature of price series requires that \nthe transfer function of the decomposition system be nonlinear. In general, the nonlinearities \nin the transfer function are able to pick up higher order moments of the input distributions \nand perfonn higher order statistical redundancy reduction between outputs. \n\n2.2 Reference input selection \n\nIn traditional approaches to blind source separation, nothing is assumed to be known about \nthe inputs, and the systems adapt on-line and without a supervisor. This works only if the \nnumber of sensors is not less than the number of independent sources. If the number of \nsensors is less than that of sources, the sources can, in theory, be separated into disjoint \ngroups (Cao & Liu 1996). However, the problem is ill-conditioned for most of the above \npractical approaches which only consider the case where the number of sensors is equal to \nthe number of sources. \n\nIn our task to decompose the multiple components of price quotes, the problem can be \ndivided into two cases. If the prices are sampled at regular intervals, we can use price \nquotes observed in different markets, and have the number of sensors be equal to the \nnumber of price components. However, in the high frequency markets, the price quotes \nare not regularly spaced in time. Price quotes from different markets will not appear at the \nsame time, so we cannot apply the price quotes from different markets to the system. In \nthis case, other reference inputs are needed. \n\nMotivated by the use of reference inputs for noise canceling (Widrow, Glover, McCool, \nKaunitz, Williams, Heam, Zeidler, Dong & Goodlin 1975), we generate three reference \ninputs from original price quotes. They are the estimates of the three desired components. \nIn the following, we briefly describe our procedure for generating the reference inputs. \nBy modeling the price quotes using a \"True Price\" state space model (Moody & Wu 1996) \n\nq(t) = rl (t) + r3(t) , \n\n(2) \n\nwhere rl (t) is an estimate of the infonnation component (True Price) and r3( t) is an estimate \nof the inventory effect component (additive noise), and by assuming that the True Price \nrl (t) is a fractional Brownian motion (Mandelbrot & Van Ness 1968), we can estimate \nrl (t) and r3 (t) with given q( t), (Moody & Wu 1996), as \n\nm,n \n\n(3) \n\n(4) \n\n\f998 \n\nL. Wu and J. E. Moody \n\nFigure 2: Multi-effect decompositions for two segments of the DEMIUSD (log prices) \nextracted from September 1995. The three panels in each segment display the observed \nprices (the dotted curve in upper panel), the regular information component (solid curve \nin upper panel), the surprise information component (mid panel) and the inventory effect \ncomponent (lower panel). \n\nwhere t/!::'(t) is an orthogonal wavelet function, Q::' is the coefficient of the wavelet \ntransform of q(t), m is the index of the scales and n is the time index of the components in \nthe wavelet transfer, S(m,B) is a smoothing function, and its parameters can be estimated \nusing the EM algorithm (Womell & Oppenheim 1992). \n\nWe then estimate the surprise information component as the residual between the informa(cid:173)\ntion component and its moving average: \n\nr2(t) = r}(t) - set) \n\n(5) \n\nset) is an exponential moving average of rl(t) and \n\nset) = (1 + a)rl(t) - as(t - 1) \n\n(6) \nwhere a is a factor. Although it can be optimized based on the training data, we set \na = -0.9 in our current work. \nOur system diagram for multi-effect decomposition is shown in Figure 1. Using multi(cid:173)\nscale decomposition Eqn(3) and smoothing techniques Eqn(6), we obtain three reference \ninputs. We can then separate the reference inputs into three independent components via \nindependent component analysis using an artificial neural network. Figure 2 presents multi(cid:173)\neffect decompositions for two segments of the DEMIUSD rates. The first segment contains \nsome impulses, and the corresponding surprise information component is able to catch \nsuch volatile movements. The second segment is basically down trending, so its surprise \ninformation component is comparatively flat. \n\n3 Empirical Analysis \n\n3.1 Mutually Independent Analysis \n\nMutual independence of the variables is satisfied if the joint probability density function \nequals the product of the marginal densities, or equivalently, the characteristic function \nsplits into the sum of marginal characteristic functions: g(X) = E7=1 gi(Xi). Taking the \nTaylor expansion of both sides of the above equation, products between different variables \nXi in the left-hand side must be zero since there are no such terms in the right-hand side. \n\n\fMulti-effect Decompositions for Financial Data Modeling \n\n999 \n\nTable 1: Comparisons between the correlation coefficients p (nonnalized) and the cross(cid:173)\ncumulants r (unnonnalized) of order 4 before and after independent component analysis \n(lCA). The DEMIUSD quotes for September 1995 is divided into 147 sub-sets of 1024 \nticks. The results presented here are the median values. The last column is the absolute \nratio of before ICA and after ICA. We note that all ratios are greater than I, indicating that \nafter ICA, the components become more independent. \n\nComponents \n\npairs \n\nCross-\n\nCumulants \n\nPl(t) \"\" P2(t) \n\npl(t) \"\" P3(t) \n\nP2(t) \"\" P3(t) \n\nP12 \nr13 \nr 22 \nr31 \nP13 \nr13 \nr22 \nr31 \nP23 \nr13 \nr22 \nr 31 \n\nBefore \nICA \n0.56 \n2.7e-14 \n-5.6e-15 \n2.0e-11 \n0.15 \n2.le-15 \n-2.0e-15 \n5.ge-12 \n0.17 \n9.le-16 \n1.2e-15 \n3.6e-15 \n\nAfter \nICA \n0.14 \n7.8e-17 \n9.2e-16 \n1.3e-13 \n0.03 \n1.6e-17 \n-4.5e-16 \n6.ge-14 \n0.04 \n-5.0e-19 \n4.ge-17 \n3.0e-17 \n\nAbsolute \nratio \n4.1 \n342.2 \n6.0 \n148.5 \n4.7 \n128.9 \n4.5 \n84.5 \n4.3 \n1806.0 \n24.3 \n119.6 \n\nWe observe the cross-cumulants of order 4: \n\nM13 - 3M2oMll \n\nr13 \nr 22 = M22 - M20M02 - 2M?1 \nr 31 = M31 - 3M02Mll \n\n(7) \n(8) \n(9) \nwhere M lei = E { x~ x~} denote the moments of order k + I. If x i and x j are independent, \nthen their cross-cumulants must be zero (Comon 1994). Table I compares the cross(cid:173)\ncumulants before and after independent component analysis (ICA) for the DEMIUSD in \nSeptember 1995. For reference, the correlation coefficients before and after ICA are also \nlisted in the table. We see that after ICA, the components have become less correlated and \nthus more independent. \n\n3.2 Autocorrelation Analysis \n\nFigure 3 depicts the autocorrelation functions of the changes in individual components \nand compares them to the original returns. We compute the short-run autocorrelations for \nthe lags up to 50. Figure 3 gives the means and standard deviations for September 1995. \nFrom the figure, we can see that both the inventory effect component and the original \nreturns show very similar autocorrelation functions, which are dominated by the significant \nnegative, first-order autocorrelations. The mean values for the other orders are basically \nequal to zero. The autocorrelations of the regular information component and the surprise \ninformation component show positive correlations except at first order. These non-zero \nautocorrelations are hidden by noise in the original series. The autocorrelation function \nof the surprise information component decays faster than that of the regular information \ncomponent. On average, it is below the 95% confidence band for lags larger than 20 ticks. \n\nThe above autocorrelation analysis suggests the following. (I) Price changes due to the \ninformation effects are slightly trending on tick-by-tick time scales. The trend in the surprise \ninformation component is shorter term than that in the regular information component. \n\n\f1000 \n\nL. Wu and J. E. Moody \n\n0.6 \n\n0.4 \na 0.2 \n~ 0 mm.mmmmHHim!lH!ml!l)mtmm \n\u00ab \n\n-0.2 \n\n-0.4 \n\n0.6 \n\n0.4 \n\n~ 0.2 \n~ 0 \n\" \u00ab -0.2 \n-0.4 \n\n0.6 \n\n0.4 \n\n~ 0.2 \n~ 0 \n\" \u00ab -0.2 \n-0.4 \n\n0.6 \n\n0.4 \n\n~ 0.2 \n~ 0 \n\" \u00ab -0.2 \n-0.4 \n\n0 \n\n10 \n\n20 \n\n30 \n\n40 \n\n0 \n\n10 \n\n20 \n\n30 \n\n40 \n\nFigure 3: Comparison of autocorrelation functions of the changes in the original observed \nprices (the upper-left panel), the inventory effect component (the lower-left panel), the regu(cid:173)\nlar information component (the upper-right panel) and the surprise infonnation component \n(the lower-right panel). The results presented are means and standard deviations, and the \nhorizontal dotted lines represent the 95% confidence band. The DEMIUSD in September \n1995 is divided into 293 sub-sets of 1024 ticks with overlapping of 512 ticks. \n\n(2) The autocorrelation function of original returns reflects only the price changes due to \nthe inventory effect. This further confirms that the existence of short tenn memory can \nmislead the analysis of dependence on longer time scales. Subsequently, we can see the \nusefulness of the multi-effect decomposition. Our empirical results should be viewed as \npreliminary, since they may depend upon the choice of True Price model. Additional \nstudies are ongoing. \n\n4 Conclusion and Discussion \n\nWe have developed a neural-net-based independent component analysis (ICA) for the \nmulti-effect decomposition of high frequency financial data. Empirical results with foreign \nexchange rates have demonstrated that the decomposed components are mutually indepen(cid:173)\ndent. The obtained regular infonnation component has recovered the trending behavior of \nthe intrinsic price movements. \n\nPotential applications for multi-effect decompositions include: \n(1) outlier detection and filtering: Filtering techniques for removing various noisy effects \nand identifYing long tenn trends have been widely studied (see for example Assimakopou(cid:173)\nlos (1995\u00bb. MuIti-effect decompositions provide us with an alternative approach. As \ndemonstrated in Section 3, the regular infonnation component can, in most cases, catch \nrelatively stable and longer tenn trends originally embedded in the price quotes. \n(2) devolatilization: Price series are heteroscedastic (Boilers lev, Chou & Kroner 1992). \nDevolatilization has been widely studied (see, for example, Zhou (1995\u00bb. The regular \ninfonnation component obtained from our multi-effect decomposition appears less volatile, \nand furthennore, its volatility changes more smoothly compared to the original prices. \n(3) mixture of local experts modeling: In most cases, one might be interested in only stable, \nlong tenn trends of price movements. However, the surprise infonnation and inventory ef(cid:173)\nfect components are not totally useless. By decomposing the price series into three mutually \n\n\fMulti-effect Decompositionsfor Financial Data Modeling \n\n1001 \n\nindependent components, the prices can be modeled by a mixture of local experts (Jacobs, \nJordan & Barto 1990), and better modeling perfonnances can be expected. \n\nReferences \nAmari, S., Cichocki, A. & Yang. H. (1996), A new learning algorithm for blind signal separation, \nin D. Touretzky. M. Mozer & M. Hasselmo, eds. 'Advances in Neural Infonnation Processing \nSystems 8', MIT Press: Cambridge, MA. \n\nAssimakopoulos. V. (1995), 'A successive filtering technique for identifying long-tenn trends'. Jour(cid:173)\n\nnal of Forecasting 14, 35-43. \n\nBarlow, H. (1961), Possible principles underlying the transfonnation of sensory messages, in \n\nW. Rosenblith, ed., 'Sensory Communication', MIT Press: Cambridge, MA, pp. 217-234. \n\nBell, A. & Sejnowski, T. (1995), 'An infonnation-maximization approach to blind separation and \n\nblind deconvolution', Neural Computation 7(6), 1129-1159. \n\nBollerslev, T., Chou, R. & Kroner. K. (1992). 'ARCH modelling in finance: A review of the theory \n\nand empirical evidence', Journal of Econometrics 8, 5-59. \n\nCao, X. & Liu, R. (1996), 'General approach to blind source separation' \u2022 IEEE Transactions on Signal \n\nProcessing 44(3), 562-569. \n\nComon, P. (1994), 'Independent component analysis, a new concept?', Signal Process 36,287-314. \nGuilaumet, D., Dacorogna, M., Dave, R., Muller, U., Olsen, R. & Pictet, O. (1994), From the bird's \neye to the microscope, a survey of new stylized facts of the intra-daily foreign exchange markets, \nTechnical Report DMG.1994-04-06, Olsen & Associates, Zurich, Switzerland. \n\nJacobs. R., Jordan, M. & Barto, A. (1990), Task decomposition through competition in a modular \nconnectionist architecture: The what and where vision tasks, Technical Report COINS 90-27, \nDepartment of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, \nMA. \n\nJutten, C. & Herault, 1. (1991). 'Blind separation of sources. part I: An adaptive algorithm based on \n\nneuromimetic architecture' , Signal Process 24( 1), 1-10. \n\nLinsker, R. (1989), An application of the principle of maximum infonnation preservation to linear \nsystems, in D. Touretzky, ed., 'Advances in Neural Infonnation Processing Systems 1', Morgan \nKaufmann Publishers, San Francisco, CA. \n\nMandelbrot, B. (1963), 'The variation of certain specUlative prices' , Journal of Business 36, 394--419. \nMandelbrot, B. & Van Ness, J. (1968), 'Fractional Brownian motion, fractional noise. and applica(cid:173)\n\ntions', SIAM Review 10. \n\nMoOdy, 1. & Wu. L. (1995), Statistical analysis and forecasting of high frequency foreign exchange \nrates, in 'The First International Conference on High Frequency Data in Finance', Zurich, \nSwitzerland. \n\nMoody, J. & Wu, L. (1996), What is the 'True Price '? - state space models for high frequency financial \ndata, in S. Amari, L. Xu, L. Chan, I. King & K. Leung, eds, 'Progress in Neural Infonnation \nProcessing (Proceedings of ICONIPS*96, Hong Kong)', Springer-Verlag, Singapore. pp. 697-\n704, Vo1.2. \n\nO'Hara, M. (1995), Market Microstructure Theory, Blackwell Business. \nPearl mutter, B. A. & Parra, L. (1997), Maximum likelihood blind source separation: a context(cid:173)\nsensitive generalization ofICA, in M. Mozer, M. Jordan & T. Petsche, eds, 'Advances in Neural \nInfonnation Processing Systems 9' , MIT Press: Cambridge, MA. \n\nWidrow, B., Glover, J., McCool, 1., Kaunitz, J., Williams, C., Hearn, R., Zeidler, J., Dong, E. & \nGoodlin, R. (1975), 'Adaptive noise cancelling: principles and applications', Proceedings of \nIEEE 63(12), 1692-1716. \n\nWornell, G. & Oppenheim, A. (1992), 'Estimation of fractal signals from noisy measurements using \n\nwavelets', IEEE Transactions on Signal Processing 40(3), 611-623. \n\nZhou, B. (1995), Forecasting foreign exchange rates series subject to de-volatilization, in 'The First \n\nInternational Conference on High Frequency Data in Finance', Zurich, Switzerland. \n\n\f\fPART IX \n\nCONTROL, NAVIGATION AND PLANNING \n\n\f\f", "award": [], "sourceid": 1251, "authors": [{"given_name": "Lizhong", "family_name": "Wu", "institution": null}, {"given_name": "John", "family_name": "Moody", "institution": null}]}