{"title": "Meta Learning with Relational Information for Short Sequences", "book": "Advances in Neural Information Processing Systems", "page_first": 9904, "page_last": 9915, "abstract": "This paper proposes a new meta-learning method -- named HARMLESS (HAwkes Relational Meta Learning method for Short Sequences) for learning heterogeneous point process models from a collection of short event sequence data along with a relational network. Specifically, we propose a hierarchical Bayesian mixture Hawkes process model, which naturally incorporates the relational information among sequences into point process modeling. Compared with existing methods, our model can capture the underlying mixed-community patterns of the relational network, which simultaneously encourages knowledge sharing among sequences and facilitates adaptively learning for each individual sequence. We further propose an efficient stochastic variational meta-EM algorithm, which can scale to large problems. Numerical experiments on both synthetic and real data show that HARMLESS outperforms existing methods in terms of predicting the future events.", "full_text": "Meta Learning with Relational Information\n\nfor Short Sequences\n\nYujia Xie\n\nCollege of Computing, Georgia Tech\n\nXie.Yujia000@gmail.com\n\nHaoming Jiang\n\nCollege of Engineering, Georgia Tech\n\njianghm@gatech.edu\n\nFeng Liu\n\nFlorida Atlantic University\n\nFLIU2016@fau.edu\n\nTuo Zhao\n\nCollege of Engineering, Georgia Tech\n\ntuo.zhao@isye.gatech.edu\n\nInstitute for Data and Decision Analytics, the Chinese University of Hong Kong, Shenzhen\n\nShenzhen Institute of Arti\ufb01cial Intelligence and Robotics for Society\n\nHongyuan Zha0\n\nzhahy@cuhk.edu.cn\n\nAbstract\n\nThis paper proposes a new meta-learning method \u2013 named HARMLESS (HAwkes\nRelational Meta LEarning method for Short Sequences) for learning heterogeneous\npoint process models from short event sequence data along with a relational net-\nwork. Speci\ufb01cally, we propose a hierarchical Bayesian mixture Hawkes process\nmodel, which naturally incorporates the relational information among sequences\ninto point process modeling. Compared with existing methods, our model can\ncapture the underlying mixed-community patterns of the relational network, which\nsimultaneously encourages knowledge sharing among sequences and facilitates\nadaptive learning for each individual sequence. We further propose an ef\ufb01cient\nstochastic variational meta expectation maximization algorithm that can scale to\nlarge problems. Numerical experiments on both synthetic and real data show that\nHARMLESS outperforms existing methods in terms of predicting the future events.\n\nIntroduction\n\n1\nEvent sequence data naturally arises in analyzing the temporal behavior of real world subjects\n(Cleeremans and McClelland, 1991). These sequences often contain rich information, which can\npredict the future evolution of the subjects. For example, the timestamps of tweets of a twitter user\nre\ufb02ect his activeness and certain state of mind, and can be used to show when he will tweet next time\n(Kobayashi and Lambiotte, 2016). The job hopping history of a person usually suggests when he will\nhop next time (Xu et al., 2017b). Unlike usual sequential data such as text data, event sequences are\nalways asynchronous and tend to be noisy (Ross et al., 1996). Therefore specialized algorithms are\nneeded to learn from such data.\nIn this paper, we are interested in short sequences, a type of sequence data that commonly appears\nin many real-world applications. Such data is usually short for two possible reasons. One is that\nthe event sequences are short in nature, such as the job hopping history. Another is the observation\nwindow is narrow. For example, we are interested in the criminal incidents of an area after a speci\ufb01c\nregulation is published. Moreover, this kind of data usually appears as a collection of sequences,\nsuch as the timestamps of many user\u2019s tweets. Our goal is to extract information that can predict the\noccurrence of future events from a large collection of such short sequences.\n\n0Corresponding author. On leave from College of Computing, Georgia Institute of Technology\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fMany existing literature considers medium-length or long sequences. They \ufb01rst model a sequence as\na parametric point process, e.g., Poisson process, Hawkes process or their neural variants, and apply\nmaximum likelihood estimation to \ufb01nd the optimal parameters (Ogata, 1999; Rasmussen, 2013).\nHowever, for short sequences, their lengths are insuf\ufb01cient for reliable inference. One remedy is\nthat we treat the collection of short sequences as independent identically distributed realizations of\nthe same point process, since many subjects, e.g., Twitter users, often share similar behaviors. This\nmakes the inference manageable. However, the learned pattern can be highly biased against certain\nindividuals, especially the non-mainstream users, since this method ignores the heterogeneity within\nthe collection.\nAn alternative is to recast the problem as a multitask learning problem (Zhang and Yang, 2017) \u2013\nwe target at multi-sequence analysis for multi-subjects. For each sequence, we consider a point\n\nprocess model that slightly deviates from a common point process model, i.e., efj = f0 + fj, where\nf0 is the common model that captures the main effect, efj is the model for the j-th sequence, and fj\n\nis the relatively small deviation. Such an assumption that there exists a universal common model\ncross all subjects, however, is still strong, since the subjects\u2019 patterns can differ dramatically. For\nexample, the job hopping history of a software engineer and a human resource manager should\nhave distinct characteristics. Furthermore, such method ignores the relationship of the subjects that\nusually can be revealed by side information. For example, a social network often shows community\npattern (Girvan and Newman, 2002) \u2013 across the communities the variation of the subjects is large,\nwhile within the communities the variation is small. The connections in the social network, such as\n\"follow\" or retweet relationship in Twitter data, can provide us valuable information to identify such\ncommunity pattern, but the aforementioned methods do not take into account such understanding to\nhelp analyzing subjects\u2019 behavior.\nTo this end, we propose a HAwkes Relational Meta LEarning method for Short Sequence (HARM-\nLESS), which can adaptively learn from a collection of short sequence. More speci\ufb01cally, in a social\nnetwork, each user often has multiple identities (Airoldi et al., 2008). For example, a Twitter user can\nbe both a military fan and a tech fan. Both his tweet history and social connections are based on his\nidentities. Motivated by above facts, we model each sequence as a hierarchical Bayesian mixture of\nHawkes processes \u2013 the weights of each Hawkes process are determined jointly by the hidden pattern\nof sequences and the relational information, e.g., social graphs.\nWe then propose a variational meta expectation maximization algorithm to ef\ufb01ciently perform\ninference. Different from existing fully bayesian inference methods (Box and Tiao, 2011; Rasmussen,\n2013; Xu and Zha, 2017), we make no assumption on the prior distribution of the parameters of\nHawkes process. Instead, when inferring for the Hawkes process parameters of the same identity\nfor all the subjects, we perform a model-agnostic adaptation from a common model for this identity\n(Finn et al. (2017), see section 3 for more details). This is more \ufb02exible since it does not restrict to a\nspeci\ufb01c form. We apply HARMLESS to both synthetic and real short event sequences, and achieve\ncompetitive performance.\nNotations: Throughout the paper, the unbold letters denote vectors or scalars, while the bold letters\ndenote the corresponding matrices or sequences. We refer the k-th entry of vector ai as ai,k. We refer\nthe i-th subject as subject i.\n2 Preliminaries\nWe brie\ufb02y introduce Hawkes Process and Model-Agnostic Meta Learning.\nHawkes processes (Hawkes, 1971) is a doubly stochastic temporal point process H(\u2713) with condi-\ntional intensity function  = (t; \u2713, \u2327 ) de\ufb01ned as\n\n(t; \u2713, \u2327 ) = \u00b5 + X\u2327 (j)<t\n\ng(t  \u2327 (j); \u21e0),\n\nwhere \u2713 = {\u00b5, \u21e0}, g is the nonnegative impact function with parameter \u21e0, \u00b5 is the base intensity, and\n\u2327 = {\u2327 (1),\u2327 (2),\u00b7\u00b7\u00b7 ,\u2327 (M )} are the timestamps of the events occurring in a time interval [0, tend].\nFunction g indicates how past events affect current intensity. Existing works usually use pre-speci\ufb01ed\nimpact functions in parametric form, e.g., the exponential function in Rasmussen (2013); Zhou et al.\n(2013) and the power-law function in Zhao et al. (2015).\nHawkes process captures an important property of real-world events \u2013 self-exciting, i.e., the past\nevents always increase the chance of arrivals of new events. For example, selling a signi\ufb01cant quantity\n\n2\n\n\fof a stock can precipitate a trading \ufb02urry. As a result, Hawkes process has been widely used in many\nareas, e.g., behavior analysis (Yang and Zha, 2013; Luo et al., 2015), \ufb01nancial analysis (Bacry et al.,\n2012), and social network analysis (Blundell et al., 2012; Zhou et al., 2013).\nModel-Agnostic Meta Learning (MAML, Finn et al., 2017) considers a set of tasks =\n{T1,T2,\u00b7\u00b7\u00b7 ,TN}, where each of the tasks only contains a very small amount of data which is\nnot enough to train a model. We want to exploit the shared structure of the tasks, to obtain models\nthat can perform well on each of the tasks. Speci\ufb01cally, MAML seeks to train a common model for\nall tasks. From optimization perspective, MAML solves the following problem,\n\nmin\n\n\u2713 XTi2\n\nFTi(e\u2713i) , min\n\n\u2713 XTi2\n\nFTi(\u2713  \u2318D(FTi,\u2713 )),\n\n(1)\n\nwhere D(\u00b7,\u00b7) is an operator, FTi is the loss function of task Ti, \u2713 is the parameter of the common\nmodel, and \u2318 is the step size. Here, D(FTi,\u2713 ) represents one or a small number of gradient update of\n\u2713. For example, in cases of one gradient step, we take D(FTi,\u2713 ) = r\u2713FTi(\u2713). This optimization\nproblem aims to \ufb01nd the common model that is expected to produce maximally effective behavior on\nthat task after performing update \u2713  \u2318D(FTi,\u2713 ).\nSolving (1) using gradient descent involves computing the Hessian matrices, which is computationally\nprohibitive. To alleviate the computational burden, First Order MAML (FOMAML) (Finn et al.,\n2017) and Reptile (Nichol et al., 2018) are then proposed. FOMAML drops the second order term in\nthe gradient of (1). Reptile further simpli\ufb01es the computation by relaxing the original update with\nHessian as a multi-step stochastic gradient descent updates. All three algorithms can be written in the\nform of (1) with operator D de\ufb01ned differently for different methods. Due to space limit, we defer\nthe de\ufb01nition of D to Appendix B.\n3 HAwkes Relational Meta LEarning for Short Sequences (HARMLESS)\n\nWe next introduce the meta learning method\nfor analyzing short sequences. Suppose we\nare given a collection of sequences T =\n{\u23271, \u23272 \u00b7\u00b7\u00b7 , \u2327N}. We also know some extra\nrelational information about the subjects. For\nexample, in social networks, we can have in-\nformation on who is friend of whom; in crim-\ninal data, we have the locations of the crimes,\nand crimes happen near each other often have\nGranger causality. Such relational information\ncan be described as a graph G = (E,V), where\nE is the node set, V is the edge set. Denote its\nadjacency matrix as Y .\nSuch social graphs often exhibit community pat-\nterns (Girvan and Newman, 2002; Xie et al.,\n2013). Within the communities the variation of subjects are small, while across the communities\nthe variation is large. Moreover, the communities are overlapping with each other, i.e., each subject\nmay belong to multiple communities and thus have multiple identities. The behaviors of the subject\nis based on the identities. Motivated by this observation, we \ufb01rst assign each subject a sum-to-one\nidentity proportion vector \u21e1i 2 [0, 1]K, whose k-th entry represents the probability of subject i\nhaving the k-th identity. In this way, we associate each subject with multiple identities rather than a\nsingle identity so that its different aspects is captured, which is more natural and \ufb02exible.\n\nFigure 1: Illustration of the suggested model.\n\nis\n\nk ) to model the timestamps of the\nk , \u2327i). For a Hawkes\nk ), the likelihood (Laub et al., 2015) of a sequence \u2327i to appear in time interval [0, tend]\n\nFor the k-th identity of subject i, we adopt Hawkes process H(e\u2713(i)\nassociated events. Denote the conditional intensity function of H(e\u2713(i)\nprocess H(e\u2713(i)\nk ; \u2327i) = exp\u21e3 Z tend\nk , \u2327i)dt + X\u2327j <tend\nL(e\u2713(i)\n\nk ) as (t;e\u2713(i)\nlog (\u2327j;e\u2713(i)\n\nk , \u2327i)\u2318.\n\n(t;e\u2713(i)\n\n(2)\n\n0\n\n3\n\n\fHere, the parametere\u2713(i)\n\nmodel-agnostic adaptation, which we will elaborate in next section.\nThe identity of the i-th subject is then a combination of the K\nidentities with identity proportion \u21e1i, and the models for indi-\nvidual sequences are essentially mixtures of Hawkes process\nk ; \u2327i). The likelihood for the\n\ni-th sequence \u2327i is\n\nmodels. Denote Li(e\u2713(i)\n\nk ) = L(e\u2713(i)\nKXk=1\n\np(\u2327i) =\n\nk ).\n\n\u21e1i,kLi(e\u2713(i)\n\n(3)\n\n\u2713\n\n\u21b5\n\nB\n\n\u02dc\u2713(i)\n1:K\n\n\u21e1i\n\n\u2327i\n\nzi!j\n\nzi j\n\nYij\n\nk is adapted from a common model with parameter \u2713k using a relatively small\n\nMoreover, the connections of the subjects are also based on their\nidentities. More speci\ufb01cally, for each connection to happen, one\nsubject i needs to approach another subject j, where the identities\nof subjects i, j are based on \u21e1i,\u21e1 j respectively. Based on this\nobservation, we adopt a Mixed Membership stochastic Block-\nmodel (MMB) (Airoldi et al., 2008) to model the connections of\nthe subjects. For each subjects pair (i, j), denote the identity of\nsubject i when subject i approaches subject j as random variable\nzi!j, and the identity of subject j when j is approached by i as\nzi j. The probability of zi!j represent the k-th identity is \u21e1i,k,\nand the probability of zi j represent the k-th identity is \u21e1j,k.\nThe probability of whether subject i and j have a connection is\nthen a function dependent on this two identities - the random variable representing the existence of\nconnection Yij follows Bernoulli distribution with parameter zT\ni!jBzi j, where B is a learnable\nparameter.\nGenerative process: The above model can be summarized as the following generative process.\n\nFigure 2: Probabilisitic graph of\nthe suggested model. The yel-\nlow nodes are parameters, white\nnodes are latent variables, and the\ngray nodes are observed variables.\nThe solid arrows represent proba-\nbilistic mapping, while the hol-\nlow arrows represent the deter-\nministic mapping.\n\n\u2022 For each node i,\n\n\u2013 Draw a K dimensional identity proportion vector \u21e1i \u21e0 Dirichlet(\u21b5).\n\u2013 Sample the i-th sequence \u2327i from the mixture of Hawkes processes described in (3).\n\n\u2022 For each pair of nodes i and j,\n\n\u2013 Draw identity indicator for the initiator zi!j \u21e0 Categorical(\u21e1i)\n\u2013 Draw identity indicator for the receiver zi j \u21e0 Categorical(\u21e1j)\n\u2013 Sample whether there is an edge between i and j, Yij \u21e0 Bernoulli(zT\n\ni!jBzi j).\n\nk , and B. The latent variables\n\nHere, the observed variables are \u2327i and Yij. The parameters are \u21b5,e\u2713(i)\n\nare \u21e1i, zi, zi!j and zi j. The graph model is shown in Figure 2.\n4 Variational Meta Expectation Maximization\nWe now introduce our variational meta expectation maximization algorithm. This algorithm incorpo-\nrates model-agnostic adaptation into variational expectation maximization. In the rest of the paper,\nwe denote z! = {zi!j}N\nTo ease the computation we add one more latent variable z. For the i-th sequence, we sample\nzi \u21e0 Categorical(\u21e1i). We regard \u2327i as a Hawkes process with parameter \u2713(i)\nzi . Note that this is\nequivalent to the mixture of Hawkes process described in previous section, since p(\u2327i) =Pk p(zi =\nk)Li(e\u2713(i)\n\nclose form.\nVariational E step. The goal is to \ufb01nd an approximation of the following posterior distribution\n\nk ). This can ease the computation because now the update for \u21e1 has\n\nzi ) =Pk \u21e1i,kLi(e\u2713(i)\n\ni,j=1,e\u2713 = {e\u2713(i)\n\ni,j=1, z = {zi j}N\n\nk }N,K\n\ni=1,k=1.\n\np(z, z!, z , \u21e1|T , Y ,\u21b5, e\u2713, B).\n\nWe aim to \ufb01nd a distribution q(z, z!, z , \u21e1) that minimizes the Kullback-Leibler (KL) divergence\nto the above posterior distribution. This can be achieved by maximizing the Evidence Lower BOund\n(ELBO, Blei et al., 2017),\n\nmax\nq2Q\n\nEq[log p(z, z!, z , \u21e1, T , Y )]  Eq[log q(z, z!, z , \u21e1)],\n\n(4)\n\n4\n\n\fwhere Q is a properly chosen distribution space. We adopt Q as the mean-\ufb01eld variational family, i.e.,\n\nq(z, z!, z , \u21e1) = q1(\u21e1)Yi\n\nq2(zi)Yj\n\nq3(zi!j)q4(zi j).\n\nwhere q1(\u21e1i) is the Probability Density Function (PDF) of Dirichlet(i), q2(zi) is the Probability\nMass Function (PMF) of Categorical(i), q3(zi!j) is the PMF of Categorical(ij), q4(zi j) is the\nPMF of Categorical( ij), and i, i, ij, ij are variational parameters. By some derivation (see\nAppendix C for detail), the updates for the variational parameters for solving problem (4) are\n\n,\n\nij,k +\n\n ij,k,\n\nk ),\n\n(5)\n\n(6)\n\nNXj=1\n\ni,k \u21b5k + i,k +\n\nNXj=1\ni,k eEq[log \u21e1i,k]Li(e\u2713(i)\nKY`=1\u21e3BYij\nij,k eEq[log \u21e1i,k]\nKYk=1(Bk`)Yij (1  Bk`)1Yijij,k , ij,` \n\nk` (1  Bk`)1Yij\u2318 ij,`\n\n ij,` eEq[log \u21e1j,`]\n\ni,kP` i,`\n\ni,k \n\n(7)\n\n, ij,k \n\nij,kP` ij,`\n ij,`Pk ij,k\nwhere Eq[log \u21e1i,k] = fdg(i,k)  fdg(P` i,`), and fdg(\u00b7) is the digamma function.\nMeta inference for \u2713 and e\u2713. Recall that the Hawkes parameter of the k-th identity of subject i is\ne\u2713(i)\nk . Instead of specifying thate\u2713(i)\nmodel H(\u2713k) to sequence i using MAML-type updates,\ne\u2713(i)\nindividual models with parametere\u2713(i)\n\n(9)\nk = \u2713k  \u2318D(log Li,\u2713 k).\nSince MAML-type algorithms only perform one or few updates from the common model, the adapted\nk within one community is close to each other, which meets our\n\nexpectation that the within-community variation should be small.\nThe gradient descent step on the log-likelihood of \u2713 can then be written as\n\nk is sampled from a prior distribution, we adapt the k-th common\n\n(8)\n\n,\n\n,\n\n\u2713k \u2713k + \u2318\u2713r\u2713k NXi=1\n\ni,k log Li(\u2713k  \u2318D(log Li,\u2713 k))! ,\n\nwhere \u2318\u2713 is the step size. In this algorithm, we only need to estimate the common models with\nparameter \u2713k, k = 1, 2,\u00b7\u00b7\u00b7 , K instead of all individual models. After we obtain \u2713k, the individual\nmodels can be easily obtained from Equation (9).\nM step. We perform maximum likelihood estimation to \u21b5 and B, The updates are as follows,\n\n(10)\n\n(11)\n\n(12)\n\n\u21b5`)  fdg(\u21b5k) +\n\nNXi=1fdg(i,k)  fdg(Xl\n\ni,`)! ,\n\n\u21b5k \u21b5k + \u2318\u21b5 Nfdg(X`\nBk` Pij Yijij,k ij,`\nPij ij,k ij,`\n\n,\n\nwhere \u2318\u21b5 is the step size. The detailed derivation can be found in Appendix C.\nAlgorithm. We perform updates (5)-(8), (10)-(12) iteratively until convergence. Note that the\nupdates can also be implemented in stochastic fashion \u2013 at each iteration, we sample a mini-batch of\nsequences, and update their associated parameters (Hoffman et al., 2013).\n5 Experiments\nWe \ufb01rst brie\ufb02y introduce oue experiment settings.\nImpact function. Following Rasmussen (2013); Zhou et al. (2013), we choose exponential impact\nfunction g(t;{, !}) = !e!t. The conditional intensity function is\n\n!e!(t\u2327 (m)),\n\n(13)\n\n(t; \u2713, \u2327 ) = (t;{\u00b5, , !}, \u2327 ) = \u00b5 + X\u2327 (m)<t\n\n5\n\n\fi\n\ni\n\nlikelihood in update (10),\n\nwhere  and ! are parameters. Note that each Hawkes process model only contains three parameters,\n\u00b5, , and !. This is because we target at short sequence. To avoid over\ufb01tting, each individual models\ncannot have too many parameters.\nRegularized likelihood function. Substitute Eq. (13) into Eq. (2), we have\n\nk ) with a regularized log-\n\nk ) , log Li(e\u2713(i)\n\nk ) + \u232b log(e\u00b5(i)\n\n!e!(\u2327 (n)\u2327 (m))\u2318\u2318.\n\nk ) + \u232bR(e\u2713(i)\nk ) , log Li(e\u2713(i)\nk ,e!(i)\nk ,e\u21b5(i)\nk = {e\u00b5(i)\n\nL(\u2713; \u2327 ) = exp\u21e3  \u00b5tend X\u2327 (n)<tend\u21e3(1  e!(tend\u2327 (n)))  log\u00b5 + X\u2327 (m)<\u2327 (n)\nTo keep the parameters non-negative, in practice we replace log Li(e\u2713(i)\nQi(e\u2713(i)\nk ) + log(e\u21b5(i)\nwheree\u2713(i)\n\nk ), (14)\nk } is the parameter of the i-th Hawkes process of the k-th identity, \u232b is a\nregularization coef\ufb01cient.\nEvaluation metric. We hold out the last timestamp of each sequence, and split the hold-out times-\ntamps into a validation set and a test set. Another option to do validation and test on event sequence\ndata is to hold out the last two timestamps \u2013 we \ufb01rst use the former ones to do validation, then train a\nnew model together with the validation timestamps, and \ufb01nally report the test result based on the later\nones. However, this is not suitable here. This is because the sequences we adopt for experiments are\nusually very short, sometimes even no more than 5 events in one sequence. As a result, the models\ntrained without or with validation timestamps, e.g., using 3 or 4 timestamps, can be signi\ufb01cantly\ndifferent, which makes the validation procedure very unreliable.\nWe report the Log-Likelihood (LL) of the test set. More speci\ufb01cally, for each sequence \u2327i =\n{\u2327 (1)\n\nk ) + log(e!(i)\n\n,\u2327 (2)\n\ni\n\n} and parameter \u2713, the likelihood of next arrival \u2327 (Mi+1)\n,\u00b7\u00b7\u00b7 ,\u2327 (Mi)\nKXk=1\ni,k\u2327 (Mi+1)\n(t;e\u2713(i)\neLi =\n\nk , \u2327i) exp\u21e3 Z \u2327 (Mi+1)\n;e\u2713(i)\n\n\u2327 (Mi)\ni\n\nk , \u2327i) dt\u2318.\n\nis\n\ni\n\ni\n\ni\n\nThe reported score is the averaged log eLi over subjects. More details can be found in Appendix D.\n\nTo estimate of the variance of the estimated log-likelihood, we adopt a multi-split procedure for\nevaluation. First, we train m candidate models with different hyper-parameters. Then we repeat\nthe following procedure for 30 times: 1). Randomly split a validation set and a test set; 2). Pick a\nmodel with highest log-likelihood on the validation set from the m candidate models; 3). Compute\nthe log-likelihood on the test set. Accordingly, we obtain 30 estimates of the log-likelihood. We then\nreport the mean and standard error of the 30 estimates.\nBaselines. We adopt four baselines as follows.\n\u21e7 MLE-Sep: We consider each sequence as a realization of an individual Hawkes process. We perform\nMaximum Likelihood Estimation (MLE) on each sequence separately, and obtain N models for N\nsequences.\n\u21e7 MLE-Com: We consider all sequences as realizations of the same Hawkes process and learn a\ncommon model by MLE.\n\u21e7 DMHP (Xu and Zha, 2017): We model sequences as a mixture of Hawkes processes with a Dirichlet\ndistribution as the prior distribution of the mixtures.\n\u21e7 MTL: We perform multi-task learning as described in Section 1. More speci\ufb01cally, we adopt\nHawkes process model for f0 and efj. Denote the parameters of f0 and efi as \u21e20 = [\u00b50, 0,! 0]T and\n\u21e2i = [\u00b5i, i,! i]T , respectively. We solve\n\nwhere k\u21e2i  \u21e20k2 is the `2 norm regularizer of \u21e2i  \u21e20 to promote the difference between f0 and fj\nto be small, \u232bmtl is a tuning parameter, and Qi(\u00b7) is the function de\ufb01ned in Eq. (14).\nWe would like to remark that another possible baseline is the hierarchical Bayesian model, i.e., we\nmodele\u2713(i)\n1:K to have prior distribution with parameter \u2713. However, such hierarchical Bayesian model\n\n6\n\nmax\n\u21e20,\u21e2i\n\n(Qi(\u21e2i) + \u232bmtlk\u21e2i  \u21e20k2) ,\n\nNXi=1\n\n\fTable 1: Visualizations of identities by HARMLESS(MAML).\nS Ground Truth\n\ndoes not have a closed-form update in variational EM algorithm. Therefore, Markov chain Monte\nCarlo should be adopted for inference, which is not scalable. For our large scale real graphs we\nconsider here, the time cost is unrealistic. Therefore we leave out this baseline.\nParameter Tuning. The detailed tuning procedure and detailed settings of each experiment can be\nfound in Appendix E.\n5.1 Synthetic Data\nData generation. We gener-\nate a dataset of 50 nodes with\nK = 6 communities. For\neach community, we generate\nHawkes meta parameters \u2713k =\n{\u00b5k, k,! k} using the follow-\ning uniform distributions:\n\u00b5k \u21e0 Uniform(0.15, 10),\nk \u21e0 Uniform(0.15, 0.85),\n!k \u21e0 Uniform(1, 10).\nWe set \u21b5 = 1K, i.e., the entries\nof \u21b5 is all one. Then for the i-th\nnode, the identity proportion \u21e1i\nis sampled from Dirichlet(\u21b5)\nand the membership indicator\nzi from the corresponding cate-\n\nK0 = 10\n\nK0 = 3\n\nK0 = 6\n\n2.0\n\n0.5\n\n1.0\n\nzi\n\n5\n\nN , 1\n\nN , 2\n\nzi by\n\ne\u00b5(i)\n\ne!(i)\n\n#{i2[1,\u00b7\u00b7\u00b7 ,N ]:zi=k}\n\nzi \u21e0 N(\u00b5zi, 0.01),\n\nzi \u21e0 N(!zi, 0.05).\n\nadding small perturbation to \u2713zi:\n\ne(i)\nzi \u21e0 N(zi, 0.01),\nN , for any k 6= `, and Bkk =\n\ngorical distribution Categorical(\u21e1i). Based on zi, we then generate the Hawkes parameterse\u2713(i)\nThe sequence is then sampled based on Hawkes process with parametere\u2713(i)\nin time interval [0, 20].\nTo ease the tuning we normalize the sequences by dividing by the largest timestamp. We set\n. We sample the graph edges based\nBk` = 0.5\non B. Denote S = Bk` \u21e5 N. The generated graphs are visualized in the second column of Table 1.\nVisualization of communities. We visualize the communities learned by HARMLESS (MAML) in\nTable 1. Denote K0 as the number of communities speci\ufb01ed in HARMLESS. We adopt K0 colors\ncorresponding to the K0 communities in the graph. The color of each node shown in the Table 1 is\nthe linear combinations of the RGB values of the K0 colors weighted by identity proportions \u21e1i.\nHARMLESS produces reasonable identities even if K0 is mis-speci\ufb01ed. If K0 < K, some of the\ncommunities would merge. If K0 > K, some of the communities would split.\nBene\ufb01t of joint training. To validate the bene\ufb01t of\njoint training on graphs and sequences, we compare\nHARMLESS result with a two step procedure: We\n\ufb01rst train an MMB model and obtain the identities,\nand train HARMLESS (MAML) with \ufb01xed identities.\nIn Figure 3 we plot the obtained log-likelihood with\nrespect to K0.\nHARMLESS (MAML) consistently achieves larger\nlog-likelihood than the two step procedure. This sug-\ngests joint training of graphs and the sequences indeed improve the prediction of future events.\nLog-likelihood with respect to K0. We also include the results of the baselines and HARMLESS\n(FOMAML) in Figure 3. The performance of HARMLESS is consistently better than the baselines.\nBesides, we \ufb01nd the performance HARMLESS (Reptile) is very dependent on the dataset. For this\nsynthetic dataset, Reptile cannot perform well.\n5.2 Real Data\nWe adopt four real datasets.\n\nFigure 3: Plot of synthetic data. S = 1.\n\n7\n\n13610K05.005.055.105.155.205.25Log-LikelihoodMLE-SepMLE-ComMTLDMHPTwo StepHARMLESS(MAML)HARMLESS(FOMAML)\fTable 2: Log-likelihood of real datasets.\n\nLinkedIn\n\nN\\A\n\n911-Calls\n\n4.0030 \u00b1 0.3763\n4.5111 \u00b1 0.3192\n4.4812 \u00b1 0.3434\n4.4621 \u00b1 0.3173\n4.5208 \u00b1 0.3256\n4.6362 \u00b1 0.3241\n4.4929 \u00b1 0.3503\n\n0.8419 \u00b1 0.0251\n0.8768 \u00b1 0.0028\n0.8348 \u00b1 0.0030\n0.9270 \u00b1 0.0027\n1.4070 \u00b1 0.0105\n1.0129 \u00b1 0.004\n0.9540 \u00b1 0.0082\n\nMathOver\ufb02ow\n0.5043 \u00b1 0.0657\n1.7805 \u00b1 0.0345\n1.5394 \u00b1 0.0347\n1.7225 \u00b1 0.0336\n1.8563 \u00b1 0.0345\n1.8344 \u00b1 0.0348\n1.8663 \u00b1 0.0342\n\nDataset\nMLE-Sep\nMLE-Com\nDMHP\nMTL\nHARMLESS (MAML)\nHARMLESS (FOMAML)\nHARMLESS (Reptile)\n\nStackOver\ufb02ow\n0.2862 \u00b1 0.0177\n1.5594 \u00b1 0.0134\n1.4910 \u00b1 0.0089\n1.3886 \u00b1 0.0082\n1.5988 \u00b1 0.0083\n1.6017 \u00b1 0.0097\n911-Calls dataset: The 911-Calls dataset1 contains emergency phone call records of \ufb01re, traf\ufb01c and\nother emergencies for Montgomery County, PA. The county is divided into disjoint areas, each of\nwhich has a unique ZIP Code. For each area, the timestamps of emergency phone calls in this area are\nrecorded as an event sequence. We consider each area as a subject, and two subjects are connected if\nthey are adjoint. We \ufb01nally obtain 57 subjects and 81 connections among them. The average length\nof the sequences is 219.1.\nLinkedIn dataset: The LinkedIn dataset (Xu et al., 2017b) contains job hopping records of the users.\nFor each user, her/his check-in timestamps corresponding to different companies are recorded as an\nevent sequence. We consider each user as a subject, and two subjects are connected if the difference in\ntimestamps of two user joined the same company is less than 2 weeks. After removing the singleton\nsubjects, we have 1, 369 subjects and 12, 815 connections among them. The average length of the\nsequences is 4.9.\nMathOver\ufb02ow dataset: The MathOver\ufb02ow dataset (Paranjape et al., 2017) contains records of the\nusers posting and answering math questions. We adopt the records from May 2, 2014 to March 6,\n2016. For each user, her/his timestamps of answering questions are recorded as an event sequence.\nWe consider each user as a subject, and two subjects are connected if one user answers another user\u2019s\nquestion. After removing the singleton subjects, we have 1, 529 subjects and 6, 937 connections\namong them. The average length of the sequences is 11.8.\nStackOver\ufb02ow dataset: StackOver\ufb02ow is a question and answer site similar to MathOver\ufb02ow. We\nadopt the records from November 8, 2015 to December 1, 2015. We construct the sequences and\ngraphs in the same way as MathOver\ufb02ow. After removing the singleton subjects, we have 13, 434\nusers and 19, 507 connections among them. The average length of the sequences is 7.7.\nResult: The log-likelihood is summarized in Table 2. Note due to Markov chain Monte Carlo is\nneeded for DMHP, we cannot get reasonable result for large dataset, i.e., StackOver\ufb02ow. HARMLESS\nperforms consistently better than the baselines. Since the standard error of the results of 911-\nCalls dataset are large, we also performed a paired t test. The test shows the difference in log-\nlikelihood between MLE-Com, i.e., best of the baselines, and HARMLESS (FOMAML), i.e., best of\nHARMLESS series, is statistically signi\ufb01cant (with p value= 1.3 \u21e5 105).\n5.3 Ablation Study\nWe then perform ablation study using\nLinkedIn dataset. Three sets of ablation\nstudy are considered here:\nRemove inner heterogeneity: We model\neach community of sequences using the\n\nTable 3: Results of ablation study.\n\nMethod\nHARMLESS (MAML)\nHARMLESS (FOMAML)\nHARMLESS (Reptile)\nRemove inner heterogeneity (K = 3)\nRemove inner heterogeneity (K = 5)\nRemove grouping (MAML)\nRemove grouping (FOMAML)\nRemove grouping (Reptile)\nRemove graph (MAML)\nRemove graph (FOMAML)\nRemove graph (Reptile)\n\nLog-Likelihood\n1.4070 \u00b1 0.0105\n1.0129 \u00b1 0.0042\n0.9540 \u00b1 0.0082\n0.9405 \u00b1 0.0032\n0.9392 \u00b1 0.0032\n0.9432 \u00b1 0.0031\n0.9376 \u00b1 0.0031\n0.9455 \u00b1 0.0041\n0.9507 \u00b1 0.0032\n0.9446 \u00b1 0.0032\n0.9489 \u00b1 0.0072\n\nk = \u2713k.\n\nsame parameters, i.e., we sete\u2713(i)\n\nRemove grouping: We set K = 1, so\nthat the whole graph is one community.\nThis equivalent to apply the MAML-type\nalgorithms on the sequences directly.\nRemove graph: We do not consider the\ngraph information, i.e., we remove z!, z , Y and B from the panel in Figure 2.\nThe results in Table 3 suggest that MAML-type adaptation, graph information, and using multiple\nidentities all contribute to the good performance of HARMLESS.\n6 Discussions\nThe setting of meta learning. The goal of conventional settings of meta learning is to train a model\non a set of tasks, so that it can quickly adapt to a new task with only few training samples. Therefore,\n\n1Data is provided by montcoalert.org.\n\n8\n\n\fpeople divide the tasks into meta training set and meta test set, where each of the task contains a\ntraining set and a test set. The meta model is trained on the meta training set, aiming to minimize the\ntest errors, and validated on the meta test set (Vinyals et al., 2016; Santoro et al., 2016). This setting\nis designed for supervised learning or reinforcement learning tasks that has accuracy or reward as a\nclear evaluation metric. Extracting information from the event sequences, however, is essentially an\nunsupervised learning task. Therefore, we do not separate meta training set and meta test set. Instead,\nwe pull the collection of tasks together, and aim to extract shared information of the collection to\nhelp the training of models on individual tasks. Here, each short sequence is a task. We exploit the\nshared pattern of the collection of the sequences to obtain the models for individual sequences.\nCommunity Pattern. The target of Mixed Membership stochastic Blockmodels (MMB) is to\nidentify the communities in a social graph, e.g., the classes in a school. However, real social graphs\ncannot always be viewed as Erd\u02ddos-R\u00e9nyi (ER) graphs assumed by MMB. As argued in Karrer and\nNewman (2011), for real-world networks, MMB tends to assign nodes with similar degrees to same\ncommunities, which is different from the popular interpretation of the community pattern. This\nproperty, however, is actually very helpful in our case. As an example, Twitter users that are more\nactive tend to have similar behavior: They tend to make more connections and post tweets more\nfrequently. In contrast, users with very different node degrees often have the tweets histories of\ndifferent characteristics, and thus should be assigned to different identities. Such property of MMB\nallows the identities in HARMLESS to represent this non-traditional community patterns in non-ER\ngraphs, i.e., it assigns subjects with various activeness to different communities.\nMixture of Hawkes processes. Many existing works adopt mixture of Hawkes process to model\nsequences that are generated from complicated mechanisms (Yang and Zha, 2013; Li and Zha, 2013;\nXu and Zha, 2017). Those works are different from HARMLESS since they do not consider the\nhierarchical heterogeneity of the sequences, and do not consider the relational information.\nVariants of Hawkes process. Some attempts have been made to further enhance the \ufb02exibility of\nHawkes processes. For example, the time-dependent Hawkes process (TiDeH) in Kobayashi and\nLambiotte (2016) and the neural network-based Hawkes process (N-SM-MPP) in Mei and Eisner\n(2017) learn very \ufb02exible Hawkes processes with complicated intensity functions. Those models\nusually have more parameters than vanilla Hawkes processes. For longer sequences, HARMLESS can\nalso be naturally extended to TiDeHs or N-SM-MPP. However, this work focuses on short sequences.\nThese methods are not useful here, since they have too many degrees of freedom.\n\nAcknowledgement\nThis work is partially supported by the grant NSF IIS 1717916 and NSF CMMI 1745382. Part of\nthe work done by Hongyuan Zha is supported by Shenzhen Institute of Arti\ufb01cial Intelligence and\nRobotics for Society, and Shenzhen Research Institute of Big Data.\n\nReferences\nACHAB, M., BACRY, E., GA\u00cfFFAS, S., MASTROMATTEO, I. and MUZY, J.-F. (2017). Uncovering\ncausality from multivariate hawkes integrated cumulants. The Journal of Machine Learning\nResearch, 18 6998\u20137025.\n\nAIROLDI, E. M., BLEI, D. M., FIENBERG, S. E. and XING, E. P. (2008). Mixed membership\n\nstochastic blockmodels. Journal of machine learning research, 9 1981\u20132014.\n\nBACRY, E., DAYRI, K. and MUZY, J.-F. (2012). Non-parametric kernel estimation for symmetric\nhawkes processes. application to high frequency \ufb01nancial data. The European Physical Journal B,\n85 157.\n\nBAUWENS, L. and HAUTSCH, N. (2009). Modelling \ufb01nancial high frequency data using point\n\nprocesses. In Handbook of \ufb01nancial time series. Springer, 953\u2013979.\n\nBENGIO, Y., BENGIO, S. and CLOUTIER, J. (1990). Learning a synaptic learning rule. Universit\u00e9\n\nde Montr\u00e9al, D\u00e9partement d\u2019informatique et de recherche ?\n\nBLEI, D. M., KUCUKELBIR, A. and MCAULIFFE, J. D. (2017). Variational inference: A review for\n\nstatisticians. Journal of the American Statistical Association, 112 859\u2013877.\n\n9\n\n\fBLUNDELL, C., BECK, J. and HELLER, K. A. (2012). Modelling reciprocating relationships with\n\nhawkes processes. In Advances in Neural Information Processing Systems.\n\nBOX, G. E. and TIAO, G. C. (2011). Bayesian inference in statistical analysis, vol. 40. John Wiley\n\n& Sons.\n\nCHALMERS, D. J. (1991). The evolution of learning: An experiment in genetic connectionism. In\n\nConnectionist Models. Elsevier, 81\u201390.\n\nCLEEREMANS, A. and MCCLELLAND, J. L. (1991). Learning the structure of event sequences.\n\nJournal of Experimental Psychology: General, 120 235.\n\nEICHLER, M., DAHLHAUS, R. and DUECK, J. (2017). Graphical modeling for multivariate hawkes\n\nprocesses with nonparametric link functions. Journal of Time Series Analysis, 38 225\u2013242.\n\nFARAJTABAR, M., YANG, J., YE, X., XU, H., TRIVEDI, R., KHALIL, E., LI, S., SONG, L. and\nZHA, H. (2017). Fake news mitigation via point process based intervention. In Proceedings of the\n34th International Conference on Machine Learning-Volume 70. JMLR. org.\n\nFARAJTABAR, M., YE, X., HARATI, S., SONG, L. and ZHA, H. (2016). Multistage campaigning in\n\nsocial networks. In Advances in Neural Information Processing Systems.\n\nFINN, C., ABBEEL, P. and LEVINE, S. (2017). Model-agnostic meta-learning for fast adaptation of\ndeep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume\n70. JMLR. org.\n\nFINN, C., XU, K. and LEVINE, S. (2018). Probabilistic model-agnostic meta-learning. In Advances\n\nin Neural Information Processing Systems.\n\nFOX, E. W., SHORT, M. B., SCHOENBERG, F. P., CORONGES, K. D. and BERTOZZI, A. L. (2016).\nModeling e-mail networks and inferring leadership using self-exciting point processes. Journal of\nthe American Statistical Association, 111 564\u2013584.\n\nGIRVAN, M. and NEWMAN, M. E. (2002). Community structure in social and biological networks.\n\nProceedings of the national academy of sciences, 99 7821\u20137826.\n\nGRANT, E., FINN, C., LEVINE, S., DARRELL, T. and GRIFFITHS, T. (2018). Recasting gradient-\n\nbased meta-learning as hierarchical bayes. arXiv preprint arXiv:1801.08930.\n\nHANSEN, N. R., REYNAUD-BOURET, P., RIVOIRARD, V. ET AL. (2015). Lasso and probabilistic\n\ninequalities for multivariate point processes. Bernoulli, 21 83\u2013143.\n\nHAWKES, A. G. (1971). Spectra of some self-exciting and mutually exciting point processes.\n\nBiometrika, 58 83\u201390.\n\nHOFFMAN, M. D., BLEI, D. M., WANG, C. and PAISLEY, J. (2013). Stochastic variational inference.\n\nThe Journal of Machine Learning Research, 14 1303\u20131347.\n\nKARRER, B. and NEWMAN, M. E. (2011). Stochastic blockmodels and community structure in\n\nnetworks. Physical review E, 83 016107.\n\nKOBAYASHI, R. and LAMBIOTTE, R. (2016). Tideh: Time-dependent hawkes process for predicting\n\nretweet dynamics. In Tenth International AAAI Conference on Web and Social Media.\n\nKOCH, G., ZEMEL, R. and SALAKHUTDINOV, R. (2015). Siamese neural networks for one-shot\n\nimage recognition. In ICML deep learning workshop, vol. 2.\n\nLAUB, P. J., TAIMRE, T. and POLLETT, P. K. (2015). Hawkes processes. arXiv preprint\n\narXiv:1507.02822.\n\nLI, L. and ZHA, H. (2013). Dyadic event attribution in social networks with mixtures of hawkes\nprocesses. In Proceedings of the 22nd ACM international conference on Information & Knowledge\nManagement. ACM.\n\n10\n\n\fLINDERMAN, S. and ADAMS, R. (2014). Discovering latent network structure in point process data.\n\nIn International Conference on Machine Learning.\n\nLUO, D., XU, H., ZHEN, Y., NING, X., ZHA, H., YANG, X. and ZHANG, W. (2015). Multi-task\nmulti-dimensional hawkes processes for modeling event sequences. In Twenty-Fourth International\nJoint Conference on Arti\ufb01cial Intelligence.\n\nMACLAURIN, D., DUVENAUD, D. and ADAMS, R. (2015). Gradient-based hyperparameter opti-\n\nmization through reversible learning. In International Conference on Machine Learning.\n\nMEI, H. and EISNER, J. M. (2017). The neural hawkes process: A neurally self-modulating\n\nmultivariate point process. In Advances in Neural Information Processing Systems.\n\nMUNKHDALAI, T. and YU, H. (2017). Meta networks. In Proceedings of the 34th International\n\nConference on Machine Learning-Volume 70. JMLR. org.\n\nNICHOL, A., ACHIAM, J. and SCHULMAN, J. (2018). On \ufb01rst-order meta-learning algorithms.\n\narXiv preprint arXiv:1803.02999.\n\nNICHOL, A. and SCHULMAN, J. (2018). Reptile: a scalable metalearning algorithm. arXiv preprint\n\narXiv:1803.02999.\n\nOGATA, Y. (1999). Seismicity analysis through point-process modeling: A review. In Seismicity\n\npatterns, their statistical signi\ufb01cance and physical meaning. Springer, 471\u2013507.\n\nPARANJAPE, A., BENSON, A. R. and LESKOVEC, J. (2017). Motifs in temporal networks. In\nProceedings of the Tenth ACM International Conference on Web Search and Data Mining. ACM.\nRASMUSSEN, J. G. (2013). Bayesian inference for hawkes processes. Methodology and Computing\n\nin Applied Probability, 15 623\u2013642.\n\nRAVI, S. and BEATSON, A. (2018). Amortized bayesian meta-learning.\nRAVI, S. and LAROCHELLE, H. (2016). Optimization as a model for few-shot learning.\nREYNAUD-BOURET, P., SCHBATH, S. ET AL. (2010). Adaptive estimation for hawkes processes;\n\napplication to genome analysis. The Annals of Statistics, 38 2781\u20132822.\n\nROSS, S. M., KELLY, J. J., SULLIVAN, R. J., PERRY, W. J., MERCER, D., DAVIS, R. M.,\nWASHBURN, T. D., SAGER, E. V., BOYCE, J. B. and BRISTOW, V. L. (1996). Stochastic\nprocesses, vol. 2. Wiley New York.\n\nSANTORO, A., BARTUNOV, S., BOTVINICK, M., WIERSTRA, D. and LILLICRAP, T. (2016).\nMeta-learning with memory-augmented neural networks. In International conference on machine\nlearning.\n\nSNELL, J., SWERSKY, K. and ZEMEL, R. (2017). Prototypical networks for few-shot learning. In\n\nAdvances in Neural Information Processing Systems.\n\nSUNG, F., YANG, Y., ZHANG, L., XIANG, T., TORR, P. H. and HOSPEDALES, T. M. (2018).\nIn Proceedings of the IEEE\n\nLearning to compare: Relation network for few-shot learning.\nConference on Computer Vision and Pattern Recognition.\n\nTRAN, L., FARAJTABAR, M., SONG, L. and ZHA, H. (2015). Netcodec: Community detection from\nindividual activities. In Proceedings of the 2015 SIAM International Conference on Data Mining.\nSIAM.\n\nTRIVEDI, R., FARAJTABAR, M., BISWAL, P. and ZHA, H. (2018). Dyrep: Learning representations\n\nover dynamic graphs.\n\nVINYALS, O., BLUNDELL, C., LILLICRAP, T., WIERSTRA, D. ET AL. (2016). Matching networks\n\nfor one shot learning. In Advances in neural information processing systems.\n\nXIE, J., KELLEY, S. and SZYMANSKI, B. K. (2013). Overlapping community detection in networks:\n\nThe state-of-the-art and comparative study. Acm computing surveys (csur), 45 43.\n\n11\n\n\fXU, H., LUO, D., CHEN, X. and CARIN, L. (2017a). Bene\ufb01ts from superposed hawkes processes.\n\narXiv preprint arXiv:1710.05115.\n\nXU, H., LUO, D. and ZHA, H. (2017b). Learning hawkes processes from short doubly-censored event\nsequences. In Proceedings of the 34th International Conference on Machine Learning-Volume 70.\nJMLR. org.\n\nXU, H. and ZHA, H. (2017). A dirichlet mixture model of hawkes processes for event sequence\n\nclustering. In Advances in Neural Information Processing Systems.\n\nYANG, S.-H. and ZHA, H. (2013). Mixture of mutually exciting processes for viral diffusion. In\n\nInternational Conference on Machine Learning.\n\nZAREZADE, A., KHODADADI, A., FARAJTABAR, M., RABIEE, H. R. and ZHA, H. (2017). Corre-\nlated cascades: Compete or cooperate. In Thirty-First AAAI Conference on Arti\ufb01cial Intelligence.\nZHANG, Y. and YANG, Q. (2017). A survey on multi-task learning. arXiv preprint arXiv:1707.08114.\nZHAO, Q., ERDOGDU, M. A., HE, H. Y., RAJARAMAN, A. and LESKOVEC, J. (2015). Seismic: A\nself-exciting point process model for predicting tweet popularity. In Proceedings of the 21th ACM\nSIGKDD International Conference on Knowledge Discovery and Data Mining. ACM.\n\nZHOU, K., ZHA, H. and SONG, L. (2013). Learning social infectivity in sparse low-rank networks\n\nusing multi-dimensional hawkes processes. In Arti\ufb01cial Intelligence and Statistics.\n\n12\n\n\f", "award": [], "sourceid": 5251, "authors": [{"given_name": "Yujia", "family_name": "Xie", "institution": "Georgia Institute of Technology"}, {"given_name": "Haoming", "family_name": "Jiang", "institution": "Georgia Institute of Technology"}, {"given_name": "Feng", "family_name": "Liu", "institution": "Florida Atlantic University"}, {"given_name": "Tuo", "family_name": "Zhao", "institution": "Georgia Tech"}, {"given_name": "Hongyuan", "family_name": "Zha", "institution": "Georgia Tech"}]}