{"title": "The Cluster Description Problem - Complexity Results, Formulations and Approximations", "book": "Advances in Neural Information Processing Systems", "page_first": 6190, "page_last": 6200, "abstract": "Consider the situation where you are given an existing $k$-way clustering $\\pi$. A challenge for explainable AI is to find a compact and distinct explanations of each cluster which in this paper is using instance-level descriptors/tags from a common dictionary. Since the descriptors/tags were not given to the clustering method, this is not a semi-supervised learning situation.  We show that the \\emph{feasibility} problem of just testing whether any distinct description (not the most compact) exists is generally intractable for just two clusters. This means that unless \\textbf{P} = \\cnp,  there cannot exist an efficient algorithm for the cluster description problem. Hence, we explore ILP formulations for smaller problems and a relaxed but restricted setting that leads to a polynomial time algorithm for larger problems.  We explore several extension to the basic setting such as the ability to ignore some instances and composition constraints on the descriptions of the clusters.  We show our formulation's usefulness on Twitter data where the communities were found using social connectivity (i.e. \\texttt{follower} relation) but the explanation of the communities is based on behavioral properties of the nodes (i.e. hashtag usage) not available to the clustering method.", "full_text": "The Cluster Description Problem - Complexity\n\nResults, Formulations and Approximations\n\nIan Davidson\u2217\n\nDepartment of Computer Science\nUniversity of California - Davis\n\ndavidson@cs.ucdavis.edu\n\nAntoine Gourru\n\nUniversite de Lyon (ERIC, Lyon 2)\nantoine.gourru@univ-lyon2.fr\n\nS. S. Ravi\u2020\n\nBiocomplexity Institute\nUniversity of Virginia\nssravi0@gmail.com\n\nAbstract\n\nConsider the situation where you are given an existing k-way clustering \u03c0. A\nchallenge for explainable AI is to \ufb01nd a compact and distinct explanation of each\ncluster which in this paper is assumed to use instance-level descriptors/tags from\na common dictionary. Since the descriptors/tags were not given to the clustering\nmethod, this is not a semi-supervised learning situation. We show that the feasibility\nproblem of testing whether any distinct description (not necessarily the most\ncompact) exists is generally intractable for just two clusters. This means that\nunless P = NP, there cannot exist an ef\ufb01cient algorithm for the cluster description\nproblem. Hence, we explore ILP formulations for smaller problems and a relaxed\nbut restricted setting that leads to a polynomial time algorithm for larger problems.\nWe explore several extensions to the basic setting such as the ability to ignore\nsome instances and composition constraints on the descriptions of the clusters. We\nshow our formulation\u2019s usefulness on Twitter data where the communities were\nfound using social connectivity (i.e. follower relation) but the explanation of the\ncommunities is based on behavioral properties of the nodes (i.e. hashtag usage) not\navailable to the clustering method.\n\n1\n\nIntroduction and Motivation\n\nThere are many clustering algorithms which perform well towards their aim of \ufb01nding cohesive groups\nof instances. The Louvain method [2] consistently generates useful results for graph clustering, spatial\ndata clustering methods such as DBScan [8] are used extensively for geographical data problems and a\nplethora of clustering methods targeted towards images, documents and graphs exist [12, 22, 5, 20, 6].\nHowever, a growing need for machine learning methods is the need for explainability. Here, we\nexplore the idea of taking an existing clustering de\ufb01ned by a partition \u03c0 = {C1, C2, . . . , Ck} found\nusing data set X and explaining it using another data set Y . For example, X could be (as they are\nin our experiments) the n \u00d7 n adjacency matrix of a graph showing the structural relation between\nindividuals (e.g. the follower relation in Twitter) and Y an n \u00d7 t behavioral information matrix\nshowing how often each individual posted on each of t different hashtags. Importantly only X and\nnot Y was used to \ufb01nd the clustering; hence, this is not a semi-supervised setting. This situation\n\u2217Institute of Advanced Studies Fellow 2017-2018 at Collegium de Lyon. Supplementary material and source\n\u2020Also with Dept. of Computer Science, University at Albany \u2013 State University of New York.\n\ncode available at www.cs.ucdavis.edu/~davidson/description-clustering.\n\n32nd Conference on Neural Information Processing Systems (NeurIPS 2018), Montr\u00e9al, Canada.\n\n\fFigure 1: A simple twitter network example where there are two (red/blue) clusters/communities\nto be explained using the hashtags each individual uses. For example person A uses MAGA, person\nB CrookedHillary and so on. In this simple example (without the dotted line) the RED cluster\nis compactly covered/explained by tags {MAGA, CrookedHillary} and the BLUE cluster with\n{ImWithHer}. But instance E only having tag MAGA (indicated by the dotted line) produces a more\ncomplex example which there is no feasible solution. One approach we explore is to ignore/omit\ninstances such as E which yields an explanation RED {MAGA, CrookedHillary} and BLUE\n{ImWitHer}.\n\nwhere the clustering is completed on one view of the data (e.g. detailed satellite imagery) and needs\nto be explained using another (e.g. high level land usage information) is common in many \ufb01elds.\nThroughout this paper we describe our work using a twitter example, but envision it could be used\nin many different settings of which it is an example of the \ufb01rst. Setting #1: Clusters are found\nusing complex features which are good for clustering but poor for explanation; hence, explanation\nis via easy to understand tags. For images, clusters found using SIFT features can be explained\nvia tags based on caption information or user speci\ufb01ed keywords. For social networks, clusters\nfound on graph topology can be explained using node attributes (e.g. hash tag usage, demographics).\nFinally, clusters found on electronic healthcare records can be explained using symptoms, diseases or\nhealthcare process events [21]. Setting #2: Clusters were historically created by humans (or nature)\nbut a modern explanation is required. For instance, consider electoral maps where electoral districts\nwere grouped into constituencies for historic or geographic reasons, perhaps decades ago [15]. We\nmay now wish to explain the clustering using current demographic information of the population.\nSetting #3: Clusters are formed using private/sensitive data but explained using public data. This\nsetting is common in security problems so as to not compromise privacy [14] by describing clusters\nusing the centroids which are based on private data. This is a variant of Setting #1 as the private data\nis good for clustering but inappropriate for explanation.\nConsider the simple example shown in Figure 1 where the twitter accounts are clustered (say based\non the follower relationship) into two clusters and are to be explained using their hashtag usage.\nWe want one group of hashtags to cover all the instances in the blue cluster and another set for the\nred cluster; however, to differentiate and explain the clusters there should be no or minimal overlap\nbetween the two explanations. It is clear there is a set cover aspect to the problem, however, regular\nset cover is insuf\ufb01cient for several reasons: (i) there are multiple universes (one for each cluster) to\ncover, (ii) each universe should be covered by non-overlapping (or minimally overlapping) collection\nof tags and (iii) there may be composition constraints on the tags to be used. For example, we may\nhave background information of the form: (a) if the hashtag #MAGA is used to describe a cluster\nthen #MakeAmericanGreatAgain should also be used or (b) #MAGA cannot be used together in a\ndescriptor that has #ImWithHer. Such background information could be generated from domain\nexperts or automatically from say hashtag co-usage statistics.\nWe formulate the cluster description problem in Section 3 and show that it is intractable even for two\nclusters. We formulate an ILP version of the problem in Section 4 and observe that the number of\nvariables and constraints is linear with respect to the number of instances, tags and clusters. However,\nin practice this formulation was found to be restrictive and sometimes the explanations contain\ninconsistencies or were incomplete. We therefore explore two extensions: (i) a formulation where\nwe can ignore a given number of the instances to ensure a feasible solution and (ii) a formulation\nwhere we can add in background information. Our experiments show these formulations can handle\nclusterings of tens of thousands of instances explained using hundreds of tags on laptops. For larger\n\n2\n\nA\tB\tC\tD\t1\t2\t3\tInstance\tTag\t#MAGA\t#Trump2016\t#CrookedHillary\tE\tF\tG\t4\t5\t#ImWithHer\t#Dems2016\t\fproblems in Section 5 we explore an \u03b1 \u2212 \u03b2 relaxation for which a polynomial time algorithm exists\nwhere each cluster\u2019s description has at most \u03b1 tags and no two cluster descriptions can have more\nthan \u03b2 tags in common. Section 6 shows a variety of different uses of our method on Twitter networks.\nWe conclude in Section 7.\nThe contributions of the paper are as follows.\n(1) We formulate the novel cluster description problem as a combinatorial optimization problem to\n\ufb01nd concise but different descriptions of each cluster.\n(2) We show that the feasibility problem of just \ufb01nding any cluster description is intractable even for\ntwo clusters (Theorem 3.1).\n(3) Our basic ILP formulation models complex forms of set cover involving multiple universes (one\nfor each cluster) and extended formulations explore ignoring instances (which are in some sense\noutliers) and composition constraints on the tags. To our knowledge these are novel.\n(4) We construct a suf\ufb01cient condition when the cluster description problem is in P (see Theorem 5.1)\nand construct an algorithm for that setting (see Algorithm 1). We term this \u03b1 \u2212 \u03b2-CONS-DESC since\neach description must be at most length \u03b1 and no two descriptions can overlap by more than \u03b2 tags.\n\n2 Related Work\n\nThere is a considerable body of literature on simultaneously performing clustering and \ufb01nding an\nexplanation. Work such as conceptual clustering [11, 9] attempts to use the same features to \ufb01nd the\nclustering and explain it and hence is limited to a single view of categorical variables. More recent\nwork has explored conceptual clustering extensions using constraint programming [13], SAT [17] or\nILP [18, 19] where the objective is to \ufb01nd a clustering and explanation simultaneously. Predictive\nclustering [16] uses the features to \ufb01nd a clustering and a descriptive function for the purpose of\nprediction and is not unlike a decision tree. All of this work is different from our work since: (i) the\ngoal is to create a new clustering not explain an existing one and (ii) the explanation uses features\nwhich were used to perform the clustering. Multi-view version of conceptual-clustering style learning\nhas been studied (e.g. [4]); these algorithms also attempt to \ufb01nd a clustering but not to explain it and\ndo not scale due to the Pareto optimization formulation.\n\n3 The Cluster Description Problem\nWe are given a set S = {s1, s2, . . . , sn} of n items and a partition \u03c0 of S into k clusters C1, C2, . . .,\nCk. We are also given a universal set T of tags, and for each item si, a set ti \u2286 T of tags, 1 \u2264 i \u2264 n.\nThe goal is to \ufb01nd a subset Tj \u2286 T of tags for each cluster Cj (1 \u2264 j \u2264 k) such that all the following\nconditions are satis\ufb01ed.\n(a) For each cluster Cj and each item si \u2208 Cj, Tj has at least one of the tags in ti; formally,\n\n|Tj \u2229 ti| \u2265 1, for each si \u2208 Cj and 1 \u2264 j \u2264 k.\n\n(b) The sets T1, T2, . . ., Tk are pairwise disjoint.\nFor 1 \u2264 j \u2264 k, the set Tj will be referred to as the descriptor for cluster Cj. We will refer to\nthe above problem as the Disjoint Tag Descriptor Feasibility (DTDF) problem. In this version,\nno constraints are imposed on the number of tags in any descriptor; any collection of descriptors\nsatisfying conditions (a) and (b) above is acceptable. Later in Section 4 we will cover the Disjoint\nTag Descriptor Minimization (DTDM) problem which adds the requirement that the size of the\ndescription is minimized, that is:\n\n(c)(cid:80)\n\nj |Tj| is minimized.\n\n3.1 Complexity of DTDF\n\nThis section and the related part of the supplementary material can be skipped on a \ufb01rst reading of\nthe paper with the understanding that the following theorem implies the computational intractability\nof DTDF and hence DTDM; that is, no polynomial time algorithms can exist for them, unless P\n\n3\n\n\f= NP. All versions of DTDF can be reduced to SAT and this allows us to identify some restricted\nversions of DTDF that can be solved ef\ufb01ciently (see Section 7).\n\nTheorem 3.1 The DTDF problem is NP-complete even when the number of clusters is 2 and the tag\nset of each item is of size at most 3.\n\nProof: See supplementary material.\n\n4 An ILP Formulation for the DTDM Problem\n\nWe \ufb01rst sketch our basic formulation for the DTDM problem and then introduce enhancements.\nBasic Formulation. We are given a clustering C1, C2 . . . Ck of n instances with each instance\ndescribed by a subset of the t = |T| tags. These tags are in the n\u00d7 t matrix Y . We solve for the k \u00d7 t\nbinary matrix X where Xi,j = 1 iff cluster i is described by tag j. One objective function then is\nsimply \ufb01nd the most concise overall cluster description:\n\n(cid:88)\n\ni,j\n\nargminX\n\nXi,j\n\n(1)\n\nHence the number of variables in this version of the formulation is kt where k is the number of\nclusters and t is the number of tags.\nOur \ufb01rst basic constraint includes the set coverage requirement for each different cluster/universe.\nHere we must de\ufb01ne the matrices S1, . . . , Sk, where Sa\ni,j = 1 iff the ith instance is actually in\ncluster a and has tag j. Note that Si, 1 \u2264 i \u2264 k, can be pre-computed. Since each instance must be\nexplained/covered there will be n constraints of this type.\n\ns.t.\n\nXk,jSk\n\ni,j \u2265 1 \u2200 i \u2208 Ck, \u2200 k\n\n(2)\n\n(cid:88)\n\nj\n\nOur next basic constraint requires that the tags chosen to represent each cluster do not overlap that is\nthey must be disjoint (wj = 1) or minimally overlap (wj > 1), where wj is the maximum number of\ntimes tag j can be used in descriptors. This is simply an OR constraint and can be encoded as:\n\ns.t.\n\nXi,j \u2264 wj \u2200 j\n\n(3)\n\n(cid:88)\n\ni\n\nThere will be t constraints of this type where t is the number of tags. So overall the number of\nvariables to solve for is O(tk) and the total number of constraints is O(n + t).\nExtended Formulation. The previous formulation meets the requirements of \ufb01nding a concise and\ndifferent description of each cluster. However, in practice we found several limitations. Firstly, as the\nintractability result shows, \ufb01nding just a feasible solution is challenging and often in experiments the\nsolver did not converge to a feasible solution. Making wj larger could address this problem but then\nthe descriptions of the clusters become more similar to each other reducing their usefulness. Secondly,\nwhen the solver did return a solution, the descriptors returned for each cluster were sometimes\nincomplete or inconsistent. In the former category a cluster could be described by #MAGA but not\n#MakeAmericaGreatAgain and in the later category a cluster could be described by both #MAGA\nand #IamWithHer.\nTo address these concerns we explore three additions. The \ufb01rst two allow side-stepping the infeasibil-\nity issue by relaxing the strict requirements of the description. The \ufb01rst such addition allows one tag\nto describe multiple clusters and the second allows ignoring some instances. The third addition incor-\nporates composition constraints to ensure that the descriptions match human or machine-generated\nguidance.\nMinimizing Overlap. Rather than minimizing the total description length we can allow the same\ndescriptor/tag to describe multiple tags but attempt to minimize these overlaps. This can be achieved\nby having the objective:\n\nargminC\n\nwj\n\n(4)\n\n(cid:88)\n\nj\n\n4\n\n\fThe number of variables in this version of the formulation is still kt. It is possible to combine this\nobjective with the objective de\ufb01ned in Equation (1).\nCover-or-Forget. To our basic coverage requirement (Equation (2)) we add in the ability to forget Ii\ninstances for cluster i. To model this, we introduce the set of variables Z where zi = 1 iff instance i\nis ignored. This can be encoded by replacing Equation (2) above with Equations (5) below. Note this\nintroduces n more optimization variables, bringing the total number to O(tk + n).\n\ns.t. zi +\n\n(cid:88)\n\nj\n\ns.t.\n\nXk,jSk\n\n(cid:88)\n\ni,j \u2265 1 \u2200 i \u2208 Ck, \u2200 k\nzi \u2264 Ik \u2200i \u2208 Ck, \u2200 k\n\n(5)\n\ni\n\nComposition Constraints. To require two tags to always be used to describe the same cluster or\ntwo tags to never describe the same cluster we introduce two sets of pairs, namely Together and\nApart, which are not unlike the must-link and cannot-link constraints used in constrained clustering\n[7, 1] though the complexity results are different (see Section 7). This adds the further constraints:\n\ns.t. Xk,i + Xk,j \u2264 1 \u2200 {i, j} \u2208 Apart, \u2200 k\ns.t. Xk,i = 1 \u2192 Xk,j = 1 \u2200{i, j} \u2208 Together, \u2200 k\n\n(6)\n(7)\n\nThe latter constraint is non-linear but can easily be modeled by merging two hashtags into one which\nsimply involves merging columns in C and S.\n\n5 A Relaxed Setting and Polynomial Time Algorithm\n\nA Note on Terminology. Throughout this section, we say that certain parameters of a problem are\n\u201c\ufb01xed\" and others are \u201cnot \ufb01xed\". These notions are commonly used in complexity theory [10]. The\nreader familiar with these concepts can skip this discussion but we provide a brief review here since the\nideas are crucial in understanding our results. Consider the Minimum Set Cover (MSC) problem [10]:\ngiven a universe U = {u1, u2, . . . , un} with n elements, a collection W = {W1, W2, . . . , Wm} with\nm subsets of U and an integer k \u2264 m, is there is a subcollection W (cid:48) of W such that |W (cid:48)| \u2264 k and\nthe union of the sets in W (cid:48) is equal to U? In the version of this problem where k is not \ufb01xed (and\nwhich is NP-complete [10]), the value of k is not known a priori, so no pre-computations are possible.\nIn the \ufb01xed parameter version (which is in P), k is a known constant (such as 4); so, one may use\npre-computation.\n\n5.1 Results for the (\u03b1, \u03b2)-CONS-DESC Problem\n\nAs shown in Section 3, one of the reasons for the computational intractability of the DTDF problem\nis the requirement that the cluster descriptors be pairwise disjoint. We now consider a version of the\ndescriptor problem, which we call (\u03b1, \u03b2)-CONS-DESC (for \u201c(\u03b1, \u03b2)-constrained descriptor\" problem),\nwhere the disjointness requirement is relaxed. Here, in addition to the clusters and the tag sets for\neach instance, we have two \ufb01xed integer parameters \u03b1 and \u03b2, and the requirements are as follows: (i)\nfor each cluster Cj, the descriptor for Cj must have a nonempty intersection with the tag set of each\ninstance in Cj, (ii) each descriptor must have at most \u03b1 tags and (iii) no two descriptors may have\nmore than \u03b2 tags in common.\nWe \ufb01rst show that the (\u03b1, \u03b2)-CONS-DESC problem can be solved in polynomial time when, in\naddition to \u03b1 and \u03b2, the number k of clusters is also \ufb01xed. We next show that the condition on k\ncannot be relaxed; that is, when k is not \ufb01xed, the (\u03b1, \u03b2)-CONS-DESC problem is NP-complete even\nwhen \u03b1 = 4 and \u03b2 = 1.\n\nTheorem 5.1 The (\u03b1, \u03b2)-CONS-DESC problem can be solved in polynomial time when the number\nof clusters k is \ufb01xed. This algorithm can also handle Together and Apart composition constraints.\n\nProof: The idea is to enumerate all the possible descriptors for each cluster systematically; the \ufb01xed\nvalues of \u03b1 and k ensure that the running time of the algorithm is a polynomial function of the input\nsize. (The steps of the described in the proof are shown in Algorithm 1.)\n\n5\n\n\f\u03b1\n\nLet N denote the maximum number of tags used in any cluster. (Note that N \u2264 |T|, where T is\nthe universal set of all descriptors.) Since the descriptor for a cluster must have at most \u03b1 tags, the\n\nnumber of possible descriptors for each cluster is(cid:0)N\n\n(cid:1) = O(N \u03b1). Call a cluster descriptor valid if it\n\nsatis\ufb01es all the given Together and Apart composition constraints. Since the size of each descriptor\nis at most \u03b1, the number of constraints is O(\u03b12). Thus, checking whether all the given constraints are\nsatis\ufb01ed can be done in O(\u03b12) time. Since \u03b1 is \ufb01xed, O(\u03b12) = O(1). Suppose we choose one valid\ndescriptor for each cluster. Let (D1, D2, . . . , Dk) be a k-tuple of descriptors, where Dj is the chosen\nvalid descriptor for cluster Cj, 1 \u2264 j \u2264 k. Since we need to consider only O(N \u03b1) descriptors for\neach cluster and the number of clusters is k, the number of k-tuples of descriptors to be considered is\nO([N \u03b1]k) = O(N k\u03b1). For each such k-tuple, we can ef\ufb01ciently check whether there is any pair of\ndescriptors that share more than \u03b2 tags. If there is no such a pair, we have a solution and the algorithm\nterminates; otherwise, we discard the current k-tuple and consider the next k-tuple of descriptors. If\nnone of the k-tuples is a solution, the algorithm terminates after indicating that there is no solution.\nThe steps are shown in Algorithm 1 where we have assumed that the existence of an algorithm that\nmaintains a circular list of valid descriptors (each of size at most \u03b1) for each cluster and returns the\nnext valid descriptor from the list whenever the statement \u201cGet the next descriptor\" is executed.\n\nAlgorithm 1: Description of our Algorithm for (\u03b1, \u03b2)-CONS-DESC\nInput\n: A collection of k clusters C1, C2, . . ., Ck with tag sets for each instance in each cluster.\nOutput : A valid descriptor with at most \u03b1 tags for each cluster such that any pair of descriptors\nhave at most \u03b2 tags in common. (Please see the main text for the de\ufb01nition of a valid\ndescriptor.)\n1 for Cluster C1 do\n2\n3\n4\n\nGet the next valid descriptor D1.\nfor Cluster C2 do\n\nGet the next valid descriptor D2.\n\nGet the next valid descriptor Dk.\nLet D = (D1, D2, . . . , Dk).\nif Each pair of descriptors in D have at most \u03b2 tags in common then\n\nOutput D as the solution and stop.\n\n...\n\nfor Cluster Ck do\n\n5\n6\n7\n8\n9\n10\n11\n12\n13\n14 end\n15 Print \u201cNo solution\".\n\nend\n\nend\n\nend\n\nThe correctness is obvious since the algorithm tries all possible combinations of valid descriptors. To\nestimate the running time, note that the algorithm considers O(N k\u03b1) k-tuples of descriptors. For each\n\n(cid:1) = O(k2) pairs of descriptors. For each pair of descriptors, a simple search\n\nk-tuple, it considers(cid:0)k\n\nthat uses O(\u03b12) time is suf\ufb01cient to determine whether the pair has more than \u03b2 tags in common.\nSince \u03b1 is \ufb01xed, O(\u03b12) = O(1). Thus, the overall running time is O(N k\u03b1k2), which is polynomial\nsince k and \u03b1 are \ufb01xed.\nOur next result shows that when the number of clusters k is not \ufb01xed, the (\u03b1, \u03b2)-CONS-DESC problem\nremains NP-complete. Our proof of the following result appears in the supplement.\n\n2\n\nTheorem 5.2 When the number of clusters k is not \ufb01xed, The (\u03b1, \u03b2)-CONS-DESC problem is NP-\ncomplete even when \u03b1 = 4 and \u03b2 = 1.\n\n6 Experimental Results with Twitter Election Data\n\nA series of easy to use MATLAB functions encoding our three formulations is available at www.cs.\nucdavis.edu/~davidson/description-clustering.\n\n6\n\n\fFigure 2: A Twitter Network of the 1000 most popular accounts divided into two communities\nusing spectral clustering explained by their use of political hashtags during the 2016 US primary\nelection season. The basic formulation was too restrictive and no feasible solution exists. The\ncover-or-forget formulation \ufb01nds a solution for I1 = I2 = 5 where no users were ignored in\nthe Republican community but the following users were ignored in the Democratic community:\nZaidJilani, VictorPopeJr, TedTheZodiac. The cover-or-forget + constrained formulation\n\ufb01nds more complete results for the Republican community and more consistent results for the\nDemocratic community.\n\nIllustrative Results With Spectral Clustering. We use an experimental data set of Twitter that\nwe collected. The Twitter data was collected from 01/01/16 until 08/22/16 and covers the political\nprimary season of the United States 2016 Presidential Election. The 1000 most politically active\ntwitter users were chosen and a graph X was constructed based on their retweet behavior. That is,\nXi,j (the weight of the edge {i, j}) is the number of times node i is retweeted by node j. Also, the\n136 most used political hashtags were collected to obtain Y .\nIn this \ufb01rst experiment we use spectral clustering to divide X into just two communities and we\nfound two natural (and obvious) communities amongst follower information: pro-Democratic and\npro-Republican. Attempting to \ufb01nd two distinct explanations (with no overlap) from the 136 hash\ntags yields no feasible solution using our basic formulation. Instead we used our cover-or-forget\nformulation setting I1 = I2 = 5 so that some instances could be ignored. However, this produced\ninconsistent results when covering the pro-Democratic community as MakeAmericaGreatAgain and\nNeverTrump are used to cover the same community! We calculated the frequency of co-occurrence\nof the hashtags in the same tweet, and from the top 2% generated Together constraints and from\nthe bottom 2% generated Apart constraints. Though this method is crude, the results are promising\nbut we are sure a human could do a better job at generating these constraints. Results are shown in\nFigure 2. Interestingly the Republican community has a simpler explanation whilst the Democratic\ncommunity\u2019s explanation was longer and less focused.\nExperiments with the Louvain Method. Here we take a data set as before except that we expand\nit to have \u2248 5000 of the most popular twitter accounts who were politically active. The Louvain\n\n7\n\nBasic\t\tFormula-on\tCover-\tor-forget\tCover-\tor-forget\t+\t\tconstraints\tNo\tfeasible\tsolu-on\tRubio,\tCruzSexScandal,\tMAGA\t\tCrookedHillary,\tBuildTheWall\tNeverTrump,\tSuperTuesday,\tIowaCaucus,\t1,\t\t\tNHPrimary,\tTrumpTrain,\tAmericaFirst\tMakeAmericaGreatAgain,\ttrump2016\tRubio,\tCruzSexScandal,\tMAGA\tCrookedHillary,\tBuildTheWall\tMakeAmericaGreatAgain\tNeverTrump,\tSuperTuesday,\tIowaCaucus,\t1,\t\t\tNHPrimary,\tIamWithHer,\tHillary2016,\tBernie,\t\tDemdebate\tRepublican\t\tCommunity\tDemocra-c\tCommunity\t\fCommunity\nPro-Clinton\nPro-Sanders\nPro-Trump\nPro-Cruz\nOther\n\nDescription\nNeverTrump ImWithHer DemDebate Sanders p2\nDemsInPhilly IowaCaucus FeelTheBern DonaldTrump\nTrump SuperTuesday MakeAmericaGreatAgain\nGOPDebate Cruz Clinton Breaking\nGOP,BernieSanders\n\nTable 1: The four main communities found by the Louvain method (the \ufb01fth is an amalgamation of\nthe smaller communities) on the Retweet graph and their description using hashtags.\n\nCommunity\nJanuary-February\nMarch-April\n\nMay-June\nJuly-August\n\nDescription\nGOPDebate, Trump2016, Cruz, tcot, VoteTrump2016, Iowa, NH\nLyinTed, CruzSexScandal, BuildTheWall, Alaska, Arkansas,\nOklahoma, Texas\nBuildTheWall, Indiana, Washington\nMakeAmericaGreat, MAGA, CrookedHillary, Benghazi\n\nTable 2: The pro-Trump communities behavior explained for four pairs of months of the primary\nseason: Jan 2016 to August 2016.\n\nmethod divides the retweet network (where the edge weight is simply how often node i retweets\na message from node j). The Louvain method discovers many clusters but the 4 largest naturally\ncorrespond to: pro-Clinton, pro-Sanders, pro-Trump, pro-Cruz and Other which is a\ncombination of many small communities. We attempt to describe these communities using the 136\nmost popular political tags and allow each tag to only appear once in each cluster. The results are in\nTable 1. For the Pro-Trump, Pro-Clinton, Pro-Cruz and Pro-Sanders communities, the results are as\nexpected referring to their candidate, the opposition and slogans.\nExperiments with Evolving Behavior. Here we take the previously found Pro-Trump community\nin Figure 1 and create four versions of its behavior from its Hashtag usage in January/February, . . .,\nJuly/August. Applying our method to this setting allows us to explain the different/evolving behavior\nof one community over time. Results are shown in Table 2.\nExperiments on Scalability. Here we explore the run time of our solver on a modest computing\nplatform (single core of a 2016 MacBook Air) using the MATLAB solver (intlinprog). No doubt\nfaster computation times could be obtained by using state of the art clusters and solvers such as\nGurobi and CPLEX but we wish to explore trends on the computation time as a function of the\nsize of the number of instances and number of tags. Table 3 show results for varying number of\ninstances (left) and number of tags (right). It is important to realize that as the number of nodes in the\ngraph becomes larger but the number of tags is a constant, we need to forget more nodes to \ufb01nd a\nfeasible explanation. From these experiments we can conclude the ILP formulations are useful for\nproblems of tens of thousands of instances and hundreds of tags. For larger problems, approximation\nformulation of Section 5 would need to be used.\n\n# Nodes\n1000\n2000\n3000\n4000\n5000\n\nk\n2\n4\n8\n16\n32\n\nTime (s) Nodes Forgotten\n0.5\n1.1\n15.3\n32.6\n88.1\n\n5\n46\n56\n89\n123\n\n# Tags\n\n25\n50\n75\n100\n136\n\nk\n32\n32\n32\n32\n32\n\nTime (s) Nodes Forgotten\n232.1\n153.5\n143.1\n123.4\n88.1\n\n783\n325\n178\n155\n123\n\nFigure 3: Runtime of basic intlinprog matlab solver on a single core of a 2016 MacBrook Air. Left\ntable shows the solver time as a function of graph size and the right table as a function of tag size.\nFor the right table the number of tags is the most frequent tags to increase the likelihood of \ufb01nding a\nsolution.\n\n8\n\n\f7 Additional Results and Conclusion\n\nHere we mention some interesting but not critical results we did not have space for in the paper,\nsketch future work and conclude. There are simple suf\ufb01cient conditions when the DTDF and DTDM\nproblems can be solved ef\ufb01ciently. The conditions arise from the fact that for two clusters, the\nproblems can be easily reduced to SAT. For example, when each instance is described by just two\ntags, the reduction leads to 2-SAT, which is in P [10]. Likewise, if the treewidth of the resulting SAT\nformula is a constant, the problem can be solved ef\ufb01ciently; this follows from known results for SAT\n[3]. The intractability results for the feasibility of satisfying the together and apart constraints are\ndifferent from those for constrained clustering [1]. In particular, \ufb01nding a feasible description just for\none cluster under apart constraints is intractable (see supplementary material) where as clustering\nunder apart constraints is only intractable for any \ufb01xed k \u2265 3 [7].\nThe cluster description problem allows taking an existing clustering obtained from one data set and\nexplaining it with another. This is useful in a variety of situations. The data used to perform the\nclustering (e.g. a graph) may not be useful for explanation, the clustering may be historical (e.g.\nelectoral maps) and need a modern explanation or problems where clusters are found on public data\nand explained with private data. We formulated the feasibility problem for \ufb01nding cluster descriptors\nand established its intractability. We then explored an ILP formulation to incorporate this formulation\nand found there were some limitations: (i) many problem instances are infeasible and (ii) some\nexplanations were inconsistent and incomplete. We addressed these two concerns by adding in a\ncover-or-forget ability so that some instances can be ignored if they are too dif\ufb01cult to cover and\ncompositions constraints not unlike the must-link and cannot-link constraints used in clustering.\nOur ILP formulations scale to 10,000\u2019s of instances but not beyond. To address even larger problems,\nwe created an \u03b1 \u2212 \u03b2 relaxation of the problem which is solvable in polynomial time when \u03b1 and \u03b2\nare \ufb01xed. It requires each cluster to be described by at most \u03b1 tags and each pair of descriptions can\nhave at most \u03b2 tags in common (overlap). Our experimental results in Twitter data show promising\nresults and the usefulness of the cluster description problem. We can now use the Louvain method\n(applied to the follower graph) to \ufb01nd clusters and explain them using hashtags.\nAcknowledgments. Ian Davidson was an Institute of Advanced Studies Fellow at the Collegium de\nLyon at the time of writing and was also supported by Deep Graph Models of Functional Networks\nGrant ONR-N000141812485 and Functional Network Discovery NSF Grant IIS-1422218. S. S. Ravi\nwas supported in part by NSF DIBBS Grant ACI-1443054, NSF BIG DATA Grant IIS-1633028\nand NSF EAGER Grant CMMI-1745207. The twitter data was provided by the ERIC lab at the\nUniversity of Lyon 2 and was prepared by one of the authors (Antoine Gourru). Thanks to Yue\nWu (UC Davis) for writing the MATLAB code available at www.cs.ucdavis.edu/~davidson/\ndescription-clustering/nips18_code.\n\n9\n\n\fReferences\n[1] Sugato Basu, Ian Davidson, and Kiri Wagstaff. Constrained clustering: Advances in algorithms,\n\ntheory, and applications. CRC Press, 2008.\n\n[2] Vincent D. Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast\nunfolding of communities in large networks. Journal of statistical mechanics: theory and\nexperiment, 2008(10):P10008, 2008.\n\n[3] H. L. Bodlaender. A tourist guide through treewidth. Acta Cybernetica, 11(1-2):1\u201322, 1993.\n\n[4] Maxime Chabert and Christine Solnon. Constraint programming for multi-criteria conceptual\n\nclustering. In CP 2017, pages 460\u2013476, 2017.\n\n[5] Rita Chattopadhyay, Wei Fan, Ian Davidson, Sethuraman Panchanathan, and Jieping Ye. Joint\ntransfer and batch-mode active learning. In International Conference on Machine Learning,\npages 253\u2013261, 2013.\n\n[6] Ian Davidson. Knowledge driven dimension reduction for clustering. In IJCAI, pages 1034\u2013\n\n1039, 2009.\n\n[7] Ian Davidson and S. S. Ravi. The complexity of non-hierarchical clustering with instance and\n\ncluster level constraints. Data Min. Knowl. Discov., 14(1):25\u201361, 2007.\n\n[8] Martin Ester, Hans-Peter Kriegel, J\u00f6rg Sander, and Xiaowei Xu. A density-based algorithm\nfor discovering clusters in large spatial databases with noise. In Proceedings of the Second\nInternational Conference on Knowledge Discovery and Data Mining (KDD-96), Portland,\nOregon, USA, pages 226\u2013231, 1996.\n\n[9] Douglas H Fisher. Knowledge acquisition via incremental conceptual clustering. Machine\n\nlearning, 2(2):139\u2013172, 1987.\n\n[10] M. R. Garey and D. S. Johnson. Computers and Intractability: A Guide to the Theory of\n\nNP-completeness. W. H. Freeman & Co., San Francisco, CA, 1979.\n\n[11] John H Gennari, Pat Langley, and Doug Fisher. Models of incremental concept formation.\n\nArti\ufb01cial intelligence, 40(1-3):11\u201361, 1989.\n\n[12] Sean Gilpin, Tina Eliassi-Rad, and Ian Davidson. Guided learning for role discovery (GLRD):\nframework, algorithms, and applications. In Proceedings of the 19th ACM SIGKDD interna-\ntional conference on Knowledge discovery and data mining, pages 113\u2013121. ACM, 2013.\n\n[13] Tias Guns, Siegfried Nijssen, and Luc De Raedt. k-Pattern set mining under constraints. IEEE\n\nTransactions on Knowledge and Data Engineering, 25(2):402\u2013418, February 2013.\n\n[14] Somesh Jha, Luis Kruger, and Patrick McDaniel. Privacy preserving clustering. In European\n\nSymposium on Research in Computer Security, pages 397\u2013417. Springer, 2005.\n\n[15] L. Kotthoff, B. O. Sullivan, S. S. Ravi, and I. Davidson. Complex clustering using constraint\nprogramming: Modeling electoral map creation. In Proc. 14th International Workshop on\nConstraint Modeling and Reformulation (ModRef 2015), Cork, Ireland, pages 1\u201314, 2015.\n\n[16] Pat Langley. Elements of machine learning. Morgan Kaufmann, 1996.\n\n[17] Jean-Philippe M\u00e9tivier, Patrice Boizumault, Bruno Cr\u00e9milleux, Mehdi Khiari, and Samir Loudni.\nConstrained Clustering Using SAT. In Proc. Advances in Intelligent Data Analysis (IDA), pages\n207\u2013218, 2012.\n\n[18] Marianne Mueller and Stefan Kramer. Integer Linear Programming Models for Constrained\n\nClustering. In Proc. Discovery Science, pages 159\u2013173, 2010.\n\n[19] A. Ouali, S. Loudni, Y. Lebbah, P. Boizumault, A. Zimmermann, and L. Loukil. Ef\ufb01ciently\n\ufb01nding conceptual clustering models with integer linear programming. In IJCAI\u201916, pages\n647\u2013654, 2016.\n\n10\n\n\f[20] Buyue Qian and Ian Davidson. Semi-supervised dimension reduction for multi-label classi\ufb01ca-\n\ntion. In AAAI, volume 10, pages 569\u2013574, 2010.\n\n[21] Peter B Walker, Jacob N Norris, Anna E Tschiffely, Melissa L Mehalick, Craig A Cunningham,\nand Ian N Davidson. Applications of transductive spectral clustering methods in a military medi-\ncal concussion database. IEEE/ACM transactions on computational biology and bioinformatics,\n14(3):534\u2013544, 2017.\n\n[22] Xiang Wang, Buyue Qian, Jieping Ye, and Ian Davidson. Multi-objective multi-view spectral\nclustering via Pareto optimization. In Proceedings of the 2013 SIAM International Conference\non Data Mining, pages 234\u2013242. SIAM, 2013.\n\n11\n\n\f", "award": [], "sourceid": 3042, "authors": [{"given_name": "Ian", "family_name": "Davidson", "institution": "U.C. Davis"}, {"given_name": "Antoine", "family_name": "Gourru", "institution": "University of Lyon - 2"}, {"given_name": "S", "family_name": "Ravi", "institution": "Biocomplexity Institute"}]}