{"title": "Inferring Interaction Networks using the IBP applied to microRNA Target Prediction", "book": "Advances in Neural Information Processing Systems", "page_first": 235, "page_last": 243, "abstract": "Determining interactions between entities and the overall organization and clustering of nodes in networks is a major challenge when analyzing biological and social network data. Here we extend the Indian Buffet Process (IBP), a nonparametric Bayesian model, to integrate noisy interaction scores with properties of individual entities for inferring interaction networks and clustering nodes within these networks. We present an application of this method to study how microRNAs regulate mRNAs in cells. Analysis of synthetic and real data indicates that the method improves upon prior methods, correctly recovers interactions and clusters, and provides accurate biological predictions.", "full_text": "Inferring Interaction Networks using the IBP applied\n\nto microRNA Target Prediction\n\nHai-Son Le\n\nMachine Learning Department\nCarnegie Mellon University\n\nPittsburgh, PA, USA\nhple@cs.cmu.edu\n\nZiv Bar-Joseph\n\nMachine Learning Department\nCarnegie Mellon University\n\nPittsburgh, PA, USA\n\nzivbj@cs.cmu.edu\n\nAbstract\n\nDetermining interactions between entities and the overall organization and clus-\ntering of nodes in networks is a major challenge when analyzing biological and\nsocial network data. Here we extend the Indian Buffet Process (IBP), a nonpara-\nmetric Bayesian model, to integrate noisy interaction scores with properties of\nindividual entities for inferring interaction networks and clustering nodes within\nthese networks. We present an application of this method to study how microR-\nNAs regulate mRNAs in cells. Analysis of synthetic and real data indicates that the\nmethod improves upon prior methods, correctly recovers interactions and clusters,\nand provides accurate biological predictions.\n\n1\n\nIntroduction\n\nDetermining interactions between entities based on observations is a major challenge when analyz-\ning biological and social network data [1, 12, 15]. In most cases we can obtain information regarding\neach of the entities (individuals in social networks and proteins in biological networks) and some\ninformation about possible relationships between them (friendships or conversation data for social\nnetworks and motif or experimental data for biology). The goal is then to integrate these datasets\nto recover the interaction network between the entities being studied. To simplify the analysis of\nthe data it is also bene\ufb01cial to identify groups, or clusters, within these interaction networks. Such\ngroups can then be mapped to speci\ufb01c demographics or interests in the case of social networks or to\nmodules and pathways in biological networks [2].\nA large number of generative models were developed to represent entities as members of a number of\nclasses. Many of these models are based on the stochastic blockmodel introduced in [19]. While the\nnumber of classes in such models could be \ufb01xed, or provided by the user, nonparametric Bayesian\nmethods have been applied to allow this number to be inferred based on the observed data [9]. The\nstochastic blockmodel was also further extended in [1] to allow mixed membership of entities within\nthese classes. An alternate approach is to use latent features to describe entities. [10] proposed a\nnonparametric Bayesian matrix factorization method to learn the latent factors in relational data\nwhereas [12] presented a nonparametric model to study binary link data. All of these methods rely\non the pairwise link and interaction data and in most cases do not utilize properties of the individual\nentities when determining interactions.\nHere we present a model that extends the Indian Buffet Process (IBP) [7], a nonparametric Bayesian\nprior over in\ufb01nite binary matrices, to learn the interactions between entities with an unbounded\nnumber of groups. Speci\ufb01cally, we represent each group as a latent feature and de\ufb01ne interactions\nbetween entities within each group. Such latent feature representation has been used in the past\nto describe entities [7, 10, 12] and IBP is an appropriate nonparametric prior to infer the number\nof latent features. However, unlike IBP our model utilizes interaction scores as priors and so the\n\n1\n\n\fmodel is not exchangeable anymore. We thus extend IBP by integrating it with Markov random\n\ufb01eld (MRF) constraints, speci\ufb01cally pairwise potentials as in Ising model. MRF priors has been\ncombined with Dirichlet Process mixture models for image segmentation in a related work of Orbanz\nand Buhmann [13]. Pairwise information is also used in the distance dependent Chinese restaurant\nprocess [4] to encourage similar objects to be clustered. Our model is well suited for cases in\nwhich we are provided with information on both link structure and the outcome of the underlying\ninteractions.\nIn social networks such data can come from observations of conversation between\nindividuals followed by actions of the speci\ufb01c individuals (for example, travel), whereas in biology\nit is suited for regulatory networks as discussed below.\nWe apply our model to study the microRNA (miRNA) target prediction problem. miRNAs were\nrecently discovered as a class of regulatory RNA molecules that regulate the levels of messenger\nRNAs (mRNAs) (which are later translated to proteins) by binding and inhibiting their speci\ufb01c\ntargets [15]. They were shown to play an important role in a number of diseases including cancer,\nand determining the set of genes that are targeted by each miRNA is an important question when\nstudying these diseases. Several methods were proposed to predict targets of miRNAs based on\ntheir sequence1. While these predictions are useful, due to the short length of miRNAs, they lead\nto many false positives and some false negatives [8].\nIn addition to sequence information, it is\nnow possible to obtain the expression levels of miRNAs and their predicted mRNA targets using\nmicroarrays. Since miRNAs inhibit their direct targets, integrating sequence and expression data\ncan improve predictions regarding the interactions between miRNAs and their targets. A number of\nmethods based on regression analysis were suggested for this task [8, 17]. While methods utilizing\nexpression data improved upon methods that only used sequence data, they often treated each target\nmRNA in isolation. In contrast, it has now been shown that each miRNA often targets hundreds\nof genes, and that miRNAs often work in groups to achieve a larger impact [14]. Thus, rather than\ntrying to infer a separate regression model for each mRNA we use our IBP extended model to infer\na joint regression model for a cluster of mRNAs and the set of miRNAs that regulate them. Such a\nmodel would provide statistical con\ufb01dence (since it combines several observations) while adhering\nmore closely to the underlying biology. In addition to inferring the interactions in the dataset such a\nmodel would also provide a grouping for genes and miRNAs which can be used to improve function\nprediction.\n\n2 Computational model\n\nFirstly, we derive a distribution on in\ufb01nite binary matrices starting with a \ufb01nite model and taking the\nlimit as the number of features goes to in\ufb01nity. Secondly, we describe the application of our model\nto the miRNA target prediction problem using a Gaussian additive model.\n\n2.1\n\nInteraction model\n\nLet zik denote the (i, k) entry of a matrix Z and let zk denote the kth column of Z. The group\nmembership of N entities is de\ufb01ned by a (latent) binary matrix Z where zik = 1 if entity i belongs\nto group k. Given Z, we say that entity i interacts with entity j if zikzjk = 1 for some k. Note that\ntwo entities can interact through many groups where each group represents one type of interaction.\nIn many cases, a prior on such interactions can be obtained. Assume we have a N \u00d7 N symmetric\nmatrix W, where wij indicates the degree that we believe that entity i and j interact: wij > 0 if\nentities i and j are more likely to interact and wij < 0 if they are less likely to do so.\nNonparametric prior for Z: Grif\ufb01ths and Ghahramani [7] proposed the Indian Buffet Process (IBP)\nas a nonparametric prior distribution on sparse binary matrices Z. The IBP can be derived from a\nsimple stochastic process, described by a culinary metaphor. In this metaphor, there are N customers\n(entities) entering a restaurant and choosing from an in\ufb01nite array of dishes (groups). The \ufb01rst\ncustomer tries Poisson(\u03b1) dishes, where \u03b1 is a parameter. The remaining customers enter one after\nthe others. The ith customer tries a previously sampled dish k with probability mk\ni , where mk is the\nnumber of previous customers who have sampled this dish. He then samples a Poisson( \u03b1\ni ) number of\nnew dishes. This process de\ufb01nes an exchangeable distribution on the equivalence classes of Z, which\nare the set of binary matrices that map to the same left-ordered binary matrices. [7]. Exchangeability\n\n1Genes that are targets of miRNAs contain the reverse complement of part of the miRNA sequence.\n\n2\n\n\fmeans that the order of the customers does not affect the distribution and that permutation of the data\ndoes not change the resulting likelihood.\nThe prior knowledge on interactions discussed above (encoded by W) violates the exchangeability\nof the IBP since the group membership probability depends on the identities of the entities whereas\nexchangeability means that permutation of entities does not change the probability. In [11], Miller\net al. presented the phylogenetic Indian Buffet Process (pIBP), where they used a tree representation\nto express non-exchangeability. In their model, the relationships among customers are encoded as\na tree allowing them to exploit the sum-product algorithm in de\ufb01ning the updates for an MCMC\nsampler, without signi\ufb01cantly increasing the computational burden when performing inference.\nWe combine the IBP with pairwise potentials using W, constraining the dish selection of customers.\nSimilar to the pIBP, the entries in zk are not chosen independently given \u03c0k but rather depend on\nthe particular assignment of the remaining entries. In the following sections, we start with a model\nwith a \ufb01nite number of groups and consider the limit as the number of groups grows to derive the\nnonparametric prior. Note that in our model, as in the original IBP [7], while the number of rows are\n\ufb01nite, the number of columns (features) could be in\ufb01nite. We can thus de\ufb01ne a prior on interactions\nbetween entities (since their number is known in advance) while still allowing for an in\ufb01nite number\nof groups. This \ufb02exibility allows the group parameters to be drawn from an in\ufb01nite mixtures of\npriors which may lead to identical groups of entities each with a different set of parameters.\n\nP (zk|\u03c0k) = exp\n\n\u03c0k|\u03b1 \u223c Beta(cid:0) \u03b1\n\n, 1(cid:1)\n\nK\n\n2.1.1 Prior on \ufb01nite matrices Z\nWe have an N \u00d7 K binary matrix Z where N is the number of entities and K is a \ufb01xed, \ufb01nite\nnumber of groups. In the IBP, each group/column k is associated with a parameter \u03c0k, chosen from\na Beta(\u03b1/K, 1) prior distribution where \u03b1 is a hyperparameter:\n\n(cid:16)(cid:88)\n\n(cid:1)(cid:17)\n(cid:0)(1 \u2212 zik) log(1 \u2212 \u03c0k) + zik log \u03c0k\n(cid:17)\n\u2212 1(cid:1) log \u03c0k\n\n(cid:1) +(cid:0) \u03b1\n\n(cid:16)(cid:88)\n\ni\n\n1\nB( \u03b1\nK , 1)\n\nThe joint probability of a column k and \u03c0k in the IBP is:\nP (zk, \u03c0k|\u03b1) =\nexp\nwhere B(\u00b7) is the Beta function.\nFor our model, we add the new pairwise potentials on memberships of entities. De\ufb01ning \u03a6zk =\n\n(cid:0)(1 \u2212 zik) log(1 \u2212 \u03c0k) + zik log \u03c0k\n(cid:1), the joint probability of a column k and \u03c0k is:\n(cid:0)(1 \u2212 zik) log(1 \u2212 \u03c0k) + zik log \u03c0k\n\n\u2212 1(cid:1) log \u03c0k\n\nexp(cid:0)(cid:80)\n\n(cid:1) +(cid:0) \u03b1\n\ni<j wijzikzjk\n1\nZ(cid:48) \u03a6zk exp\n\nP (zk, \u03c0k|\u03b1) =\n\n(cid:16)(cid:88)\n\n(cid:17)\n\n(1)\n\n(2)\n\nK\n\nK\n\ni\n\nwhere Z(cid:48) is the partition function. Note that IBP is a special case of our model when all w\u2019s are\nzeros (W = 0).\nFollowing [7], we de\ufb01ne the lof-equivalence classes [Z] as the sets of binary matrices mapped to\nthe same left-ordered binary matrices. The history hi of a feature k at an entity i is de\ufb01ned as\n(z1k, . . . , z(i\u22121)k). When no object is speci\ufb01ed, h refers to the full history. mk and mh denote the\nnumber of non-zero entries of a feature k and a history h respectively. Kh is the number of features\nh=1 Kh is\n\npossessing the history h while K0 is the number of features having mk = 0. K+ =(cid:80)2N\u22121\n\nthe number of features for which mk > 0.\nBy integrating over all values of \u03c0k, we get the marginal probability of a binary matrix Z.\n\ni\n\n(cid:90) 1\n\nk=1\n\nK(cid:89)\nK(cid:89)\nK(cid:89)\n\nk=1\n\nk=1\n\nP (Z) =\n\n=\n\n=\n\nP (zk, \u03c0k|\u03b1) d\u03c0k\n\n0\n\n1\nZ(cid:48) \u03a6zk\n\n(cid:90) 1\nZ(cid:48) \u03a6zk B(cid:0) \u03b1\n\n1\n\n0\n\nK\n\nexp\n\n(cid:16)(cid:0) \u03b1\n+ mk, N \u2212 mk + 1(cid:1)\n\nK\n\n3\n\n(cid:17)\n+ mk \u2212 1(cid:1) log \u03c0k + (N \u2212 mk) log(1 \u2212 \u03c0k)\n\n(3)\n\nd\u03c0k\n\n(4)\n\n(5)\n\n\fThe partition function Z(cid:48) could be written as: Z(cid:48) =(cid:80)2N\u22121\n\nh=0 \u03a6hB(cid:0) \u03b1\n\nK + mh, N \u2212 mh + 1(cid:1).\n\n2.1.2 Taking the in\ufb01nite limit\n\n(cid:88)\n\nThe probability of a particular lof-equivalence class of binary matrices, [Z], is:\n\nZ(cid:48) \u03a6zk B(cid:0)mk +\nTaking the limit when K \u2192 \u221e, we can show that with \u03a8 =(cid:80)2N\u22121\n\nh=0 Kh!\n\nP ([Z]) =\n\nP (Z) =\n\n\u03b1\nK\n\nk=1\n\nK(cid:89)\n\n1\n\nZ\n\n, N \u2212 mk + 1(cid:1)\n\n(N\u2212mh)!(mh\u22121)!\n\nK!(cid:81)2N\u22121\nK+(cid:89)\n\nh=0 Kh!\n\n\u03b1K+(cid:81)2N\u22121\n\n=\n\nh=1 Kh!\n\nk=1\n\nh=1 \u03a6h\nK , N \u2212 mk + 1)\nK , N + 1)\n\nK(cid:89)\nexp(cid:0) \u2212 \u03b1\u03a8)\n\nk=1\n\nN !\n\n1\nZ(cid:48) B(\n\n\u03b1\nK\n\n\u03a6zk\n\nB(mk + \u03b1\nB( \u03b1\n\n(N \u2212 mk)!(mk \u2212 1)!\n\nN !\n\n\u03a6zk\n\nK!(cid:81)2N\u22121\nK+(cid:89)\n\nk=1\n\nK\u2192\u221e P ([Z]) = lim\nlim\nK\u2192\u221e\n\n(6)\n\n:\n\n, N + 1) (7)\n\n(8)\n\nThe detailed derivations are shown in Appendix.\n\n2.1.3 The generative process\n\nWe now describe a generative stochastic process for Z. It can be understood by a culinary metaphor,\nwhere each row of Z corresponds to a customer and each column corresponds to a dish. We denote\nby h(i) the value of zik in the complete history h. With \u00af\u03a6h = \u03a6h\n, we de\ufb01ne\n\n(N\u2212mh)!(mh\u22121)!\n\ni=1 \u03a8i. Finally, let z<ik be entries 1, . . . , (i \u2212 1) of zk.\n\nN !\n\n\u00af\u03a6h so that \u03a8 =(cid:80)N\n\n\u03a8i =(cid:80)\n\nh:hi=0,h(i)=1\n\ncustomers. The probability that a dish would be selected is(cid:80)\n\nAssume that we are provided with a compatibility score between pairs of customers. That is, we have\na value wij for the food preference similarity between customer i and customer j. Higher values of\nwij indicate similar preferences and customers with such values are more likely to select the same\ndish. Therefore, the dishes a customer selects may depend on the choices of previous customers. The\n\ufb01rst customer tries Poisson(\u03b1\u03a81) dishes. The remaining customers enter one after the others. The\nith customer selects dishes with a probability that partially depends on the selection of the previous\n\u00af\u03a6h.\nHe then samples a Poisson(\u03b1\u03a8i) number of new dishes. This process repeats until all customers\nhave made their selections. Although this process is not exchangeable, the sequential order of cus-\ntomers is not important. This means that we get the same marginal distribution for any particular\norder of customers. Let K (i)\n1 denote the number of new dishes sampled by customer i, the probabil-\nity of a particular matrix generated by this process is:\n\n\u00af\u03a6h/(cid:80)\n\nh:hi=z<ik,h(i)=1\n\nh:hi=z<ik\n\nK+(cid:89)\n\n\u00af\u03a6zk exp(cid:0) \u2212 \u03b1\u03a8)\n\n\u03b1K+(cid:81)N\n\ni=1 K (i)\n\n1\n\nk=1\n\nP (Z) =\n\n(9)\n\n(cid:81)N\n(cid:81)2N \u22121\n\ni=1 K(i)\nh=1 Kh!\n\nmatrices gen-\nIf we only pay attention to the lof-equivalence classes [Z], since there are\nerated by this process mapped to the same equivalence classes, multiplying P (Z) by this quantity\nrecovers Equation (8). We show in Appendix that in the case of the IBP where \u03a6h = 1 for all\nhistories h (when W = 0), this generative process simpli\ufb01es to the Indian Buffet Process.\n\n1\n\n2.2 Regression model for mRNA expression\n\nIn this section, we describe the application using the nonparametric prior to the miRNA target pre-\ndiction problem. However, the method is applicable in general settings where there is a way to model\nproperties of one entity from properties of its interacting entities. Our input data are expression pro-\n\ufb01les of M messenger RNA (mRNA) transcripts and R miRNA transcript across T samples. Let\nM )T , where each row vector xi is the expression pro\ufb01le of mRNA i in all samples.\nX = (xT\nSimilarly, let Y = (yT\nR)T represent the expression pro\ufb01les of R miRNAs. Furthermore,\nsuppose we are given a M \u00d7 R matrix C where cij is the prior likelihood score for the interaction of\n\n1 , . . . , xT\n\n1 , . . . , yT\n\n4\n\n\fmRNA i and miRNA j. Such matrix C could be obtained from sequence-based miRNA target pre-\ndictions as discussed above. Applying our interaction model to this problem, the set of N = M + R\nentities are divided into two disjoint sets of mRNAs and miRNAs. Let Z = (UT , VT )T where U\nand V are the group membership matrices for mRNAs and miRNAs respectively, W is given by\n. Therefore, mRNA i and miRNA j interact through all groups k such that uikvjk = 1.\n\n(cid:18) 0 C\n(cid:19)\n\nCT\n\n0\n\n2.2.1 Gaussian additive model\n\nIn the interaction model suggested by GenMiR++ [8], each miRNA expression pro\ufb01le is used to\nexplain the downregulation of the expression of its targeted mRNAs. Our model uses a group speci\ufb01c\nand miRNA speci\ufb01c coef\ufb01cients ( s = [s1, . . . , s\u221e]T , with sk > 0 for groups and r = [r1, . . . , rR]T\nfor all miRNAs) to model the downregulation effect. These coef\ufb01cients represent the baseline effect\nof group members and the strength of speci\ufb01c miRNAs, respectively. Using these parameters the\nexpression level of a speci\ufb01c mRNA could be explained by summing over expression pro\ufb01les of all\nmiRNAs targeting the mRNA:\n\n(cid:88)\n\n(rj +\n\nk:uikvjk=1\n\nxi \u223c N(cid:0)\u00b5 \u2212(cid:88)\n(cid:16) \u2212 1\n\nj\n\nk:uikvjk=1 sk) yj.\n\nsk) yj, \u03c32I(cid:1)\n(cid:17)\n\n(10)\n\n(11)\n\nr \u223c\n\nwhere \u00b5 represents baseline expression for this mRNA and \u03c3 is used to represent measurement noise.\nThus, under this model, the expression of a mRNA are reduced from their baseline values by a linear\ncombination of expression values of the miRNAs that target them. The probability of the observed\ndata given Z is: P (X, Y|Z, \u0398) \u221d exp\n, with \u0398 = {\u00b5, \u03c32, s, r}\n\n(cid:80)\ni(xi \u2212 \u00afxi)T (xi \u2212 \u00afxi)\n\n2\u03c32\n\nand \u00afxi = \u00b5 \u2212(cid:80)\n\nj(rj +(cid:80)\n\n2.2.2 Priors for model variables\n\nWe use the following as prior distributions for the variables in our model:\nr \u223c N (0, \u03c32\n\nr I)\n\nsk \u223c Gamma(\u03b1s, \u03b2s)\n\u00b5 \u223c N (0, \u03c32\n\n\u00b5I)\n\n1/\u03c32 \u223c Gamma(\u03b1v, \u03b2v)\n\nwhere the \u03b1 and \u03b2 are the shape and scale parameters. The parameters are given hyperpriors: 1/\u03c32\n\u00b5 \u223c Gamma(a\u00b5, b\u00b5). \u03b1s, \u03b2s, \u03b1v, \u03b2v are also given Gamma hyperpriors.\nGamma(ar, br) and 1/\u03c32\n\n3\n\nInference by MCMC\n\nAs with many nonparametric Bayesian models, exact inference is intractable.\nInstead we use a\nMarkov Chain Monte Carlo (MCMC) method to sample from the posterior distribution of Z and \u0398.\nAlthough, our model allows Z to have in\ufb01nite number of columns, we only need to keep track of\nnon-zero columns of Z, an important aspect which is exploited by several nonparametric Bayesian\nmodels [7]. Our sampling algorithm involves a mix of Gibbs and Metropolis-Hasting steps which\nare used to generate the new sample.\n\n3.1 Sampling from populated columns of Z\n\nLet m\u2212ik is the number of one entries not including zik in zk. Also let z\u2212ik denote the entries of\nzk except zik and let Z\u2212(ik) be the entire matrix Z except zik. The probability of an entry given\nthe remaining entries in a column can be derived by considering an ordering of customers such that\ncustomer i is the last person in line and using the generative process in Section 2.1.3:\n\nP (zik = 1|z\u2212ik) =\n\n=\n\n=\n\n\u00af\u03a6z<ik,zik=1 + \u00af\u03a6z<ik,zik=0\n\n\u00af\u03a6z<ik,zik=1\n\nj(cid:54)=i wijzjk\n\n(cid:1)(N \u2212 m\u2212ik \u2212 1)!m\u2212ik!\n\nexp(cid:0)(cid:80)\n(cid:1)(N \u2212 m\u2212ik \u2212 1)!m\u2212ik! + (N \u2212 m\u2212ik)!(m\u2212ik \u2212 1)!\n(cid:1)m\u2212ik + (N \u2212 m\u2212ik)\n\n(cid:1)m\u2212ik\n\nj(cid:54)=i wijzjk\n\nexp(cid:0)(cid:80)\n\nj(cid:54)=i wijzjk\n\nj(cid:54)=i wijzjk\n\nexp(cid:0)(cid:80)\nexp(cid:0)(cid:80)\n\n5\n\n\fWe could also get the result using the limiting probability in Equation (8). The probability of each\nzik given all other variables is: P (zik|X, Y, Z\u2212(ik)) \u221d P (X, Y|Z\u2212(ik), zik)P (zik|z\u2212ik). We need\nonly to condition on z\u2212ik since columns of Z are generated independently.\n\n3.2 Sampling other variables\n\nSampling a new column of Z: New columns are columns that do not yet have any entries equal\nto 1 (empty groups). When sampling for an entity i, we assume this is the last customer in line.\nTherefore, based on the generative process described in Section 2.1.3, the number of new features\nare Poisson( \u03b1\nN ). For each new column, we need to sample a new group speci\ufb01c coef\ufb01cient variable\nsk. We can simply sample from the prior distribution given in Equation (11) since the probability\nP (X, Y|Z, \u0398) is not affected by these new columns since no interactions are currently represented\nby these columns.\nSampling sk for populated columns: Since we do not have a conjugate prior on s, we cannot com-\npute the conditional likelihood directly. We turn to Metropolis-Hasting to sample s. The proposed\ndistribution of a new value s\u2217\nh ) where h is the\nshape parameter. The mean of this distribution is the old value sk. The acceptance ratio is\n\nk given the old value sk is q(s\u2217\n\n(cid:104)\n\nA(sk \u2192 s\u2217\n\nk) = min\n\n1,\n\nk|sk) = Gamma(h, sk\nk|\u03b1s, \u03b2s) q(sk|s\u2217\nk)\n\nk) p(s\u2217\n\n(cid:105)\n\nP (X, Y|Z, \u0398 \\ {sk}, s\u2217\n\nP (X, Y|Z, \u0398) p(sk|\u03b1s, \u03b2s) q(s\u2217\n\nk|sk)\n\nIn our experiments, h is selected so that the average acceptance rate is around 0.25 [5].\nSampling r, \u00b5, \u03c32 and prior parameters: Closed-form formulas for the posterior distributions of\nr,\u00b5 and \u03c32 can be derived due to their conjugacy. For example, the posterior distribution of 1/\u03c32\n\n2 ,(cid:0) 1\ngiven the other variables is: Gamma(cid:0)\u03b1v + M T\n\n(cid:1)\u22121(cid:1).Equations for updates\n\ni(xi\u2212\u00afxi)T (xi\u2212\u00afxi)\n\n\u00b5 since we\nof r and \u00b5 are omitted due to lack of space. Gibbs sampling steps are used for \u03c32\ncan compute the posterior distribution with conjugate priors. For prior parameters {\u03b1s, \u03b2s, \u03b1v, \u03b2v},\nwe use Metropolis-Hasting steps discussed previously.\n\nr and \u03c32\n\n+\n\n\u03b2v\n\n(cid:80)\n\n2\n\n4 Results\n\nWe name our method GroupMiR (Group MiRNA target prediction). In this section we compare the\nperformance of GroupMiR with GenMiR++ [8], which is one of the popular methods for predict-\ning miRNA-mRNA interactions. However, unlike our method it does not use grouping of mRNAs\nand attempts to predict each one separately. Besides, there are two other important differences of\nGenMiR++ from our method: 1) GenMiR++ only consider interactions in the candidate set while\nour method consider all possible interactions. 2) GenMiR++ accepts a binary matrix as a candi-\ndate set while our method allows continuous valued scores. To our best knowledge, GenMiR++,\nwhich uses the regression model for interaction between entities, is the only appropriate method2\nfor comparison.\n\n4.1 Synthetic data\n\nWe generated 9 synthetic datasets. Each dataset contains 20 miRNAs and 200 mRNAs. We set\nthe number of groups to K = 5 and T = 10 for all datasets. The miRNA membership V is a\nrandom matrix with at most 5 ones in each column. The mRNA membership U is a random matrix\nwith density of 0.1. The expression of mRNAs are generated from the model in Equation (10) with\n\u03c32 = 1. The remaining random variables are sampled as follows: y \u223c N (0, 1), s \u223c N (1, 0.1)\nand r \u223c N (0, 0.1). Since the sequence based predictions of miRNA-mRNA interactions are based\non short complementary regions they often result in many more false positives than false negatives.\nWe thus introduce noise to the true binary interaction matrix C(cid:48) by probabilistically changing each\nzero value in that matrix to 1. We tested different noise probabilities: 0.1, 0.2, 0.4 and 0.8. We use\nC = 2C(cid:48) \u2212 1.8, \u03b1 = 1 and the hyperprior parameters are set to generic values. Our sampler is ran\nfor 2000 iterations and 1000 iterations are discarded as burn-in.\n\n2We also tested with the original IBP (by setting W = 0). The results for both the synthetic and real data\n\nwere too weak to be comparable with GenMIR++. See Appendix.\n\n6\n\n\f(a) Truth (b) 0.1\n\n(c) 0.2\n\n(d) 0.4\n\n(e) 0.8\n\nFigure 1: The posterior distribution of K.\n\nFigure 2: An example synthetic dataset.\n\nFigure 1 plots the estimated posterior distribution of K from the samples of the 9 datasets for all\nnoise levels. As can be seen, when the noise level is small (0.1), the distributions are correctly\ncentered around K = 5. With increasing noise levels, the number of groups is overestimated.\nHowever, GroupMiR still does very well at a noise level of 0.4 and estimates for the higher noise\nlevel are also within a reasonable range.\nWe estimated a posterior mean for the interaction matrix Z by \ufb01rst ordering the columns of each\nsampled Z and then selecting the mode from the set of Z matrices. GenMiR++ returns a score value\nin [0, 1] for each potential interaction. To convert these to binary interactions we tested a number\nof different threshold cutoffs: 0.5, 0.7 and 0.9. Figure 3 presents a number of quality measures\nfor the recovered interactions by the two methods. GroupMiR achieves the best F1 score across all\nnoise levels greatly improving upon GenMiR++ when high noise levels are considered (a reasonable\nbiological scenario). In general, while the precision is very high for all noise levels, recall drops to\na lower rate. From a biological point of view, precision is probably more important than recall since\neach of the predictions needs to be experimentally tested, a process that is often time consuming and\nexpensive.\nIn addition to accurately recovering interactions between miRNAs and mRNAs, GroupMiR also\ncorrectly recovers the groupings of mRNA and miRNAs. Figure 2 presents a graphical view of\nthe group membership in both the true model and the model recovered by GroupMiR for one of\nthe synthetic datasets. As can be seen, our method is able to accurately recover the groupings of\nboth miRNAs and mRNAs with moderate noise levels (up to 0.4). For the higher noise level (0.8)\nthe method assigns more groups than in the underlying model. However, most interactions are still\ncorrectly recovered. These results hold for all datasets we tested (not shown due to lack of space).\n\n(a) Recall\n\n(b) Accuracy\n\n(c) Precision\n\n(d) F1 Score\n\nFigure 3: Performance of GroupMiR versus GenMiR++: Each data point is a synthetic dataset.\n\n4.2 Application to mouse lung development\n\nTo test our method on real biological data, we used a mouse lung developmental dataset [6]. In this\nstudy, the authors used microarrays to pro\ufb01le both miRNAs and mRNAs at 7 time points, which\ninclude all recognized stages of lung development. We downloaded the log ratio normalized data\ncollected in this study. Duplicate samples were averaged and median values of all probes were\nassigned to genes. As suggested in the paper, we used ratios to the last time point resulting in\n6 values for each mRNA and miRNA. Priors for interaction between miRNA and mRNA were\ndownloaded from the MicroCosm Target3 database. Selecting genes with variance in the top 10%,\nled to 219 miRNAs and 1498 mRNAs which were used for further analysis.\nWe collected 5000 samples of the interaction matrix Z following a 5000 iteration burn-in period.\nConvergence of the MCMC chain is determined by monitoring trace plots of K in multiple chains.\n\n3http://www.ebi.ac.uk/enright-srv/microcosm/\n\n7\n\n0.10.20.40.800.20.40.60.81NoiseRate0.10.20.40.80.40.60.81NoiseRate0.10.20.40.80.20.40.60.81NoiseRate0.10.20.40.800.20.40.60.81NoiseScore\f(a)\n\n(b)\n\n(c)\n\n(d)\n\n(e)\n\n(f)\n\nFigure 4: Interaction network recovered by GroupMiR: Each node is a pie chart corresponding to\nits expression values in the 6 time points (red: up-regulation, green: down-regulation).\nSince there are many more entries for real data compared to synthetic data we computed a consensus\nfor Z by reordering columns in each sample and averaging the entries across all matrices.\nWe further analyzed the network constructed from groups with at least 90% posterior probability.\nThe network recovered by GroupMiR is more connected (89 nodes and 208 edges) when compared\nto the network recovered by GenMiR++ (using equivalent 0.9 threshold) with 37 nodes and 34 edges\n(Appendix). We used Cytoscape [16] to visualize the 6 groups of interactions in Figure 4. The\nnetwork contains several groups of co-expressed miRNAs controlling sets of mRNA, in agreement\nwith previous biological studies [20].\nTo test the function of the clusters identi\ufb01ed, we performed Gene Ontology (GO) enrichment anal-\nysis for the mRNAs using GOstat [3]. The full results (Bonferroni corrected) are presented in Ap-\npendix. As can be seen, several cell division categories are enriched in cluster (b) which is expected\nwhen dealing with a developing organ (which undergoes several rounds of cell division). Other\nsigni\ufb01cant functions include organelle organization and apoptosis which also are associated with\ndevelopment (cluster (c)). We performed similar GO enrichment analysis for the GenMiR++ results\nand for K-means when using the same set of mRNAs (setting k = 6 as in our model). In both cases\nwe did not \ufb01nd any signi\ufb01cant enrichment indicating that only by integrating sets of miRNAs with\nthe mRNAs for this data we can \ufb01nd functional biological groupings. See Appendix for details.\nWe have also looked at the miRNAs controlling the different clusters and found that in a number of\ncases these agreed with prior knowledge. Cluster (a) includes 2 members of the miR 17-92 cluster,\nwhich is known to be critical to lung organogenesis [18]. MiRNA families miR-30, miR-29, miR-20\nand miR-16, all identi\ufb01ed by our method, were also reported to play roles in the early stages of lung\norganogenesis [6]. It is important to point out that we did not \ufb01lter miRNAs explicitly based on\nexpression but these miRNAs came in the results based on their strong effect on mRNA expression.\n\n5 Conclusions\n\nWe have described an extension to IBP that allows us to integrate priors on interactions between\nentities with measured properties for individual entities when constructing interaction networks.\nThe method was successfully applied to predict miRNA-mRNA interactions and we have shown\nthat it works well on both synthetic and real data. While our focus in this paper was on a biolog-\nical problem, several other datasets provide similar information including social networking data.\nOur method is appropriate for such datasets and can help when attempting to construct interaction\nnetworks based on observations.\n\nAcknowledgments\n\nThis work is supported in part by NIH grants 1RO1GM085022, 1U01HL108642 and NSF grant\nDBI-0965316 to Z.B.J.\n\n8\n\nMyh14Slc6a14mmu-miR-106bAfpEar11Ear2Cyp2f2mmu-miR-93Ear1mmu-miR-106ammu-miR-20aTmem100Lgals3mmu-miR-17Lilrb3Parp3Dapk2Fmo1Brca1AurkbH19BlmCol24a14932413O14Rik2610318N02RikCcna2Pdgfcmmu-miR-29cTmem48Agtr2mmu-miR-29aPafah1b3Col11a16720463M24RikLin7aLmnb1Dbf4Mki67Kntc1Fkbp3Etaa1Wdr75Dhx9Diap3Tubb3PpihMsh6Fancd2Tcfl54732471D19Rikmmu-miR-30cmmu-miR-30bZranb3Igsf9Ppa1mmu-miR-30dmmu-miR-30emmu-miR-30aCenpqBrca2Ube2tMthfd1lRad54lAbcb7mmu-miR-30e*mmu-miR-30a*C330027C09RikHmmrMcm8Cdca8Gins1Dlk1mmu-miR-195Palb2ClspnWnk3mmu-miR-16Shcbp1Top2aDusp9mmu-miR-27bAurkaKif2cRrm2mmu-miR-27aNrk\fReferences\n[1] E.M. Airoldi, D.M. Blei, S.E. Fienberg, and E.P. Xing. Mixed membership stochastic block-\n\nmodels. The Journal of Machine Learning Research, 9:1981\u20132014, 2008.\n\n[2] Z. Bar-Joseph, G.K. Gerber, T.I. Lee, et al. Computational discovery of gene modules and\n\nregulatory networks. Nature biotechnology, 21(11):1337\u20131342, 2003.\n\n[3] T. Bei\u00dfbarth and T.P. Speed. GOstat: \ufb01nd statistically overrepresented Gene Ontologies within\n\na group of genes. Bioinformatics, 20(9):1464, 2004.\n\n[4] David M. Blei and Peter Frazier. Distance dependent chinese restaurant processes. In Johannes\n\nFrnkranz and Thorsten Joachims, editors, ICML, pages 87\u201394. Omnipress, 2010.\n\n[5] S. Chib and E. Greenberg. Understanding the metropolis-hastings algorithm. Amer. Statisti-\n\ncian, 49(4):327\u2013335, 1995.\n\n[6] J. Dong, G. Jiang, Y.W. Asmann, S. Tomaszek, et al. MicroRNA Networks in Mouse Lung\n\nOrganogenesis. PloS one, 5(5):4645\u20134652, 2010.\n\n[7] T. Grif\ufb01ths and Z. Ghahramani. In\ufb01nite latent feature models and the Indian buffet process. In\n\nAdvances in Neural Information Processing Systems, 18:475, 2006.\n\n[8] J.C. Huang, T. Babak, T.W. Corson, et al. Using expression pro\ufb01ling data to identify human\n\nmicroRNA targets. Nature methods, 4(12):1045\u20131049, 2007.\n\n[9] C. Kemp, J.B. Tenenbaum, T.L. Grif\ufb01ths, , et al. Learning systems of concepts with an in\ufb01nite\n\nrelational model. In Proc. 21st Natl Conf. Artif. Intell.(1), page 381, 2006.\n\n[10] E. Meeds, Z. Ghahramani, R.M. Neal, and S.T. Roweis. Modeling dyadic data with binary\n\nlatent factors. In Advances in NIPS, 19:977, 2007.\n\n[11] K.T. Miller, T.L. Grif\ufb01ths, and M.I. Jordan. The phylogenetic indian buffet process: A non-\n\nexchangeable nonparametric prior for latent features. In UAI, 2008.\n\n[12] K.T. Miller, T.L. Grif\ufb01ths, and M.I. Jordan. Nonparametric latent feature models for link\n\nprediction. In Advances in Neural Information Processing Systems, 2009.\n\n[13] P. Orbanz and J.M. Buhmann. Nonparametric bayesian image segmentation.\n\nJournal of Computer Vision, 77(1):25\u201345, 2008.\n\nInternational\n\n[14] ME Peter. Targeting of mrnas by multiple mirnas: the next step. Oncogene, 29(15):2161\u20132164,\n\n2010.\n\n[15] N. Rajewsky. microRNA target predictions in animals. Nature genetics, 38:S8\u2013S13, 2006.\n[16] P. Shannon, A. Markiel, O. Ozier, et al. Cytoscape: a software environment for integrated\n\nmodels of biomolecular interaction networks. Genome research, 13(11):2498, 2003.\n\n[17] F. Stingo, Y. Chen, M. Vannucci, et al. A Bayesian graphical modeling approach to microRNA\n\nregulatory network inference. Ann. Appl. Statist, 2010.\n\n[18] A. Ventura, A.G. Young, M.M. Winslow, et al. Targeted Deletion Reveals Essential and Over-\n\nlapping Functions of the miR-17-92 Family of miRNA Clusters. Cell, 132:875\u2013886, 2008.\n\n[19] Y.J. Wang and G.Y. Wong. Stochastic blockmodels for directed graphs. Journal of the Ameri-\n\ncan Statistical Association, 82(397):8\u201319, 1987.\n\n[20] C. Xiao and K. Rajewsky. MicroRNA control in the immune system: basic principles. Cell,\n\n136(1):26\u201336, 2009.\n\n9\n\n\f", "award": [], "sourceid": 183, "authors": [{"given_name": "Hai-son", "family_name": "Le", "institution": null}, {"given_name": "Ziv", "family_name": "Bar-joseph", "institution": null}]}