{"title": "On the Relation Between Low Density Separation, Spectral Clustering and Graph Cuts", "book": "Advances in Neural Information Processing Systems", "page_first": 1025, "page_last": 1032, "abstract": null, "full_text": "On the Relation Between Low Density Separation, Spectral\n\nClustering and Graph Cuts\n\nHariharan Narayanan\n\nDepartment of Computer Science\n\nUniversity of Chicago\n\nChicago IL 60637\n\nDepartment of Computer Science and Engineering\n\nMikhail Belkin\n\nThe Ohio State University\n\nColumbus, OH 43210\n\nhari@cs.uchicago.edu\n\nmbelkin@cse.ohio-state.edu\n\nPartha Niyogi\n\nDepartment of Computer Science\n\nUniversity of Chicago\n\nChicago IL 60637\n\nniyogi@cs.uchicago.edu\n\nAbstract\n\nOne of the intuitions underlying many graph-based methods for clustering and semi-supervised\nlearning, is that class or cluster boundaries pass through areas of low probability density. In this\npaper we provide some formal analysis of that notion for a probability distribution. We introduce\na notion of weighted boundary volume, which measures the length of the class/cluster boundary\nweighted by the density of the underlying probability distribution. We show that sizes of the cuts of\ncertain commonly used data adjacency graphs converge to this continuous weighted volume of the\nboundary.\n\nkeywords: Clustering, Semi-Supervised Learning\n\n1 Introduction\n\nConsider the probability distribution with density p(x) depicted in Fig. 1, where darker color denotes higher probability\ndensity. Asked to cluster this probability distribution, we would probably separate it into two roughly Gaussian bumps\nas shown in the left panel.\nSame intuition applies to semi-supervised learning. Asked to point out more likely groups of data of the same type,\nwe would be inclined to believe that these two bumps contain data points with the same labels. On the other hand,\nthe class boundary shown in the right panel seems rather less likely. One way to state this basic intuition is the Low\nDensity Separation assumption [5], saying that the class/cluster boundary tends to pass through regions of low density.\nIn this paper we propose a formal measure on the complexity of the boundary, which intuitively corresponds to the\nLow Density Separation assumption. We will show that given a class boundary, this measure can be computed from\na \ufb01nite sample from the probability distribution. Moreover, we show this is done by computing the size of a cut for a\npartition of a certain standard adjacency graph, de\ufb01ned on that sample, and point out some interesting connections to\nspectral clustering.\nTo \ufb01x our intuitions, let us consider the question of what makes the cut in the left panel more intuitively acceptable\nthan the cut in the right. Two features of the left cut make it more pleasing: the cut is shorter in length and the cut\n\n\fFigure 1: A likely cut and a less likely cut.\n\n(cid:82)\n\npasses through a low-density area. Note that a very jagged cut through a low-density area or a short cut through the\nmiddle of a high-density bump would be unsatisfactory. It will therefore appear reasonable to take the length of the\ncut as a measure of its complexity but weight it depending on the density of the probability distribution p through\nwhich it passes. In other words, we propose the weighted length of the boundary, represented by the contour integral\ncut p(s)ds to measure the complexity of a cut. It is clear that the boundary in the left panel has a\nalong the boundary\nconsiderably lower weighted length than the boundary in the right panel of our Fig. 1.\nTo formalize this notion further consider a (marginal) probability distribution with density p(x) supported on some\ndomain or manifold M. This domain is partitioned in two disjoint clusters/parts. Assuming that the boundary S is a\nS p(s)ds. Note that just as in the example above,\nsmooth hypersurface we de\ufb01ne the weighted volume of the cut to be\nthe integral is taken over the surface of the boundary.\nWe will show how this quantity can be approximated given empirical data and establish connections with some popular\ngraph-based methods.\n\n(cid:82)\n\n2 Connections and related work\n\n2.1 Spectral Clustering\n\nOver the last two decades there has been considerable interest in various spectral clustering techniques (see, e.g., [6]\nfor an overview). The idea of spectral clustering can be expressed very simply. Given a graph, we would often like\nto construct a balanced partitioning of the vertex set, i.e. a partitioning such which minimizes the number (or total\nweight) of edges across the cut. This is generally an NP-hard optimization problem. It turns out, however, that a\nsimple real-valued relaxation can be used to reduce it to standard linear algebra, typically to \ufb01nding eigenvectors of a\ncertain graph Laplacian. We note that the quality of partition is usually measured in terms of the corresponding cut\nsize.\nA critical question, when this notion is applied to general purpose clustering in the context of machine learning is how\nto construct the graph given data points. A typical choice here is the Gaussian weights (e.g., [14]). To summarize, a\ngraph is obtained from a point cloud, using Gaussian or other weights, and partitioned using spectral clustering or a\ndifferent algorithm, which attempts to approximate the smallest (balanced) cut.\nWe note that while the intuition is that spectral clustering is an approximation to the minimum cut, and is closely\nrelated to random walks and diffusions on graphs and the underlying probability distributions ([13, 12]), existing\nresults on convergence of spectral clustering ([11]) do not provide a formal interpretation of the limiting partition or\nconnect it to the size of the resulting cut.\n\n2.2 Graph-based semi-supervised learning\n\nSimilarly to spectral clustering, graph-based semi-supervised learning constructs a graph from the data. In contrast\nto clustering, however, some of the data is labeled. The problem is typically to either label the unlabeled points\n(transduction) or, more generally, to build a classi\ufb01er de\ufb01ned on the whole space. This may be done trying to \ufb01nd the\n\n2\n\n\fFigure 2: Curves of small and high condition number respectively\n\nminimum cut, which respects the labels of the data directly ([3]), or, using graph Laplacian as a penalty functional\n(e.g.,[15, 1]).\nOne of the important intuitions of semi-supervised learning is the cluster assumption(e.g., [4]) or, more speci\ufb01cally,\nthe low density separation assumption suggested in [5], which states that the class boundary passes through a low\ndensity region. We argue that this intuition needs to slightly modi\ufb01ed by suggesting that cutting through a high density\nregion may be acceptable as long as the length of the cut is very short. For example imagine two high-density round\nclusters connected by a very thin high-density thread. Cutting the thread is appropriate as long as the width of the\nthread is much smaller than the radii of the clusters.\n\n2.3 Convergence of Manifold Laplacians\n\nAnother closely related line of research is the connections between point-cloud graph Laplacians and Laplace-Beltrami\noperators on manifolds, which have been explored recently in [9, 2, 10]. A typical result in that setting shows that for\na \ufb01xed function f and points sampled from a probability distribution on a manifold or a domain, the graph Laplacian\napplied to f converges to the manifold Laplace-Beltrami operator \u2206Mf. We note that the results of those papers\ncannot be directly applied in our situation as for us f is the indicator function of a subset and is not differentiable.\nEven more importantly, this paper establishes an explicit connection between the point-cloud Laplacian applied to\nsuch characteristic functions (weighted graph cuts) and a geometric quantity, which is the weighted volume of the cut\nboundary. This geometric connection does not easily follow from results of those papers and the techniques used in\nthe proof of our Theorem 3 are signi\ufb01cantly different.\n\n3 Summary of the Main Results\nLet p be a probability density function on a domain M \u2286 Rd.\nLet S be a smooth hypersurface that separates M into two parts, S1 and S2. The smoothness of S will be quanti\ufb01ed\nby a condition number 1/\u03c4, where \u03c4 is the radius of the largest ball that can be placed tangent to the manifold at any\npoint, intersecting the manifold at only one point. It bounds the curvature of the manifold.\n\nDe\ufb01nition 1 Let Kt(x, y) be the heat kernel in Rd given by\n1\n\nKt(x, y) :=\n\n(4\u03c0t)d/2\n\ne\u2212(cid:107)x\u2212y(cid:107)2/4t.\n\n1\n\n(4\u03c0t)d/2 .\n\nLet Mt := Kt(x, x) =\nLet X := {x1, . . . , xN} be a set of N points chosen independently at random from p. Consider the complete graph\nwhose vertices are associated with the points in X, and where the weight of the edge between xi and xj, i (cid:54)= j is given\n\n3\n\n\fby\n\nWij = Kt(xi, xj)\n\nLet W be the weight matrix. Let X1 = X \u2229 S1 and X2 = X \u2229 S2 be the data point which land in S1 and S2\nrespectively. Let D be the diagonal matrix whose entries are row sums of W (degrees of the corresponding vertices)\n\n(cid:88)\n\nj\n\nDii =\n\nWij\n\nThe normalized Laplacian associated to the data X (and parameter t) is the matrix L(t, X) := I \u2212 D\u22121/2W D\u22121/2.\nLet f = (f1, . . . , fN ) be the indicator vector for X1:\n\n(cid:189)\n\nfi =\n\n1\n0\n\nif xi \u2208 X1\notherwise\n\nThere are two quantities of interest:\n\n(cid:82)\n\n1.\n\nS p(s)ds, which measures the quality of the partition S in accordance with the weighted volume of the\nboundary.\n\n2. f T Lf, which measures the quality of the empirical partition in terms of its cut size.\n\nOur main Theorem shows that after an appropriate scaling, the empirical cut size converges to the volume of the\nboundary.\nTheorem 1 Let the number of points, |X| = N tend to in\ufb01nity and {tN}\u221e\nzero such that tN > 1\n\n0 , be a sequence of values of t that tend to\n\n. Then with probability 1,\n\n1\n\n2d+2\n\nN\n\nFurther, for any \u03b4 \u2208 (0, 1) and any \u0001 \u2208 (0, 1/2), there exists a positive constant C and an integer N0(depending on \u0001,\n\u03b4 and certain generic invariants of p and S) such that with probability 1 \u2212 \u03b4, (\u2200N > N0),\n\n\u221a\n\u221a\n\u03c0\n\nN\n\nt\n\nlim\n\nf T L(tN , X)f =\n\np(s)ds\n\n(cid:175)(cid:175)(cid:175)(cid:175) \u221a\n\nN\n\n\u221a\n\u03c0\n\nf T L(tN , X)f \u2212\n\nt\n\np(s)ds\n\n(cid:175)(cid:175)(cid:175)(cid:175) < Ct\u0001\n\nN .\n\n(cid:90)\n\nS\n\n(cid:90)\n\nS\n\u221a\n\u221a\n\u03c0\nt\n\nf T L(tN , X)f to a heat \ufb02ow across the relevant cut\nThis theorem is proved by \ufb01rst relating the empirical quantity\n(on the continuous domain), and then relating the heat \ufb02ow to the measure of the cut. In order to state these results,\nwe need the following notation.\n\nN\n\nDe\ufb01nition 2 Let\n\n\u03c8t(x) =\n\nLet\n\nand\n\n\u03b1(t) :=\n\n\u03b2(t, X) :=\n\n(cid:114)\n\n(cid:90)\n\n(cid:90)\n\n\u03c0\nt\n\n(cid:113)(cid:82)\n\np(x)\n\n.\n\nM Kt(x, z)p(z)dz\n\u221a\n\u221a\n\u03c0\nt\n\nf T L(t, X)f\n\nN\n\nKt(x, y)\u03c8t(x)\u03c8t(y)dxdy.\n\nS1\n\nS2\n\nWhere t and X are clear from context, we shall abbreviate \u03b2(t, X) to \u03b2 and \u03b1(t) to \u03b1. In theorem 2 we show that for\na \ufb01xed t, as the number of points |X| = N tends to in\ufb01nity, with probability 1, \u03b2(t, X) tends to \u03b1(t).\nIn theorem 3 we show that \u03b1(t) can be made arbitrarily close to the weighted volume of the boundary by making t\ntend to 0.\n\n4\n\n\fFigure 3: Heat \ufb02ow \u03b1 tends to\n\n(cid:82)\n\nS p(s)ds\n\n(1)\n\n(2)\n\n(3)\n\nTheorem 2 Let 0 < \u00b5 < 1. Let u := 1/\np and S such that with probability greater than 1 \u2212 exp (\u2212C1 N \u00b5)\n1 + t\n\n|\u03b2(t, X) \u2212 \u03b1(t)| < C2\n\n(cid:179)\n\nd+1\n\n2\n\nu\u03b1(t).\n\n\u221a\nt2d+1N 1\u2212\u00b5. Then, there exist positive constants C1, C2 depending only on\n\nTheorem 3 For any \u0001 \u2208 (0, 1\n\n(cid:90)\n\n(cid:175)(cid:175)(cid:175)(cid:175)(cid:114)\n\n(cid:90)\n2), there exists a constant C such that for all t such that 0 < t < \u03c4(2d)\u2212 e\ne\u22121 ,\n\u03c0\nt\n\nKt(x, y)\u03c8t(x)\u03c8t(y)dxdy \u2212\n\n(cid:175)(cid:175)(cid:175)(cid:175) < C t\u0001.\n\np(s)ds\n\nS2\n\nS1\n\n(cid:180)\n(cid:90)\n\nS\n\nBy letting N \u2192 \u221e and tN \u2192 0 at suitable rates and putting together theorems 2 and 3, we obtain the following\ntheorem:\n\u221a\nt2d+1N 1\u2212\u00b5 \u2192 0.\nTheorem 4 Let the number of random data points N \u2192 \u221e, and tN \u2192 0, at rates so that u := 1/\nThen, for any \u0001 \u2208 (0, 1/2), there exist positive constants C1, C2 depending only on p and S , such that for any N > 1\nwith probability greater than 1 \u2212 exp (\u2212C1(N \u00b5)),\n\n(cid:175)(cid:175)(cid:175)(cid:175)\u03b2 (tN , X) \u2212\n\n(cid:90)\n\np(s)ds\n\nS\n\n(cid:175)(cid:175)(cid:175)(cid:175) < C2 (t\u0001 + u)\n\n4 Outline of Proofs\nTheorem 1 is a corollary of Theorem 4, obtained by setting u to be t\u0001, and setting \u00b5 to 1\u22122\u0001\n2d+2 . N0 is chosen to\nbe a large enough integer so that an application of the union bound on all N > N0, still gives us a probability\n\nexp (\u2212C1(N \u00b5)) < \u03b4, of the rate of convergence being worse than stated in Theorem 1.\n\n(cid:80)\n\nN >N0\nTheorem 4 is a direct consequence of Theorem 2 and Theorem 3.\nTheorem 2:\nWe prove theorem 2 using a generalization of McDiarmid\u2019s inequality from [7, 8]. McDiarmid\u2019s inequality asserts\nthat a function of a large number of independent random variables, that is not very in\ufb02uenced by the value of any one\nof these, takes a value close to its mean. In the generalization that we use, it is permitted that over a bad set that has\na small probability mass, the function is highly in\ufb02uenced by some of the random variables. In our setting, it can be\nshown that our measure of a cut, f T Lf is such a function of the independent random points in X, and so the result is\napplicable. There is another step involved, since the mean of f T Lf is not \u03b1, the quantity to which we wish to prove\nf T L(t, X)f] tends to \u03b1(t) as N tends to in\ufb01nity. Now,\nconvergence. Therefore we need to prove that the mean E[\n\n\u221a\n\u221a\n\u03c0\nt\n\nN\n\nf T L(t, X)f = 1/N\n\n\u03c0/t\n\n(cid:112)\n\n(cid:88)\n\n\u221a\n\u221a\n\u03c0\nt\n\nN\n\n(cid:88)\n\n(cid:80)\n\n{(\n\nx\u2208X1\n\ny\u2208X2\n\n5\n\nKt(x, y)\n\n(cid:80)\nz(cid:54)=y Kt(y, z))}1/2\n\n.\n\nz(cid:54)=x Kt(x, z))(\n\nS1/2tS1S2\fIf, instead, we had in the denominator of the right side\n\nusing the linearity of Expectation,\n\n\uf8ee\uf8ef\uf8ef\uf8ef\uf8ef\uf8f0\n\n(cid:112)\n\n(cid:88)\n\n(cid:88)\n\nx\u2208X1\n\ny\u2208X2\n\nE\n\n1\n\nN(N \u2212 1)\n\n\u03c0/t\n\np(z)Kt(x, z)dz\n\np(z)Kt(y, z)dz,\n\nM\n\nM\n\n\uf8f9\uf8fa\uf8fa\uf8fa\uf8fa\uf8fb = \u03b1(t).\n\n(cid:182)\n\np(z)Kt(y, z)dz\n\n(cid:115)(cid:181)(cid:82)\n\nM\n\nKt(x, y)\n\n(cid:182)(cid:181)(cid:82)\n\np(z)Kt(x, z)dz\n\nM\n\nUsing Chernoff bounds, we can show that with high probability, for all x \u2208 X,\n\n(cid:118)(cid:117)(cid:117)(cid:116)(cid:90)\n\n(cid:80)\n\nz(cid:54)=x Kt(x, z)\nN \u2212 1\n\n\u2248\n\np(z)Kt(x, z)dz.\n\nM\n\n(cid:90)\n\n(cid:90)\n\n(cid:82)\n\n(cid:90)\n\n(cid:90)\n\nPutting the last two facts together and using the Generalization of McDiarmid\u2019s inequality from [7, 8], the result\nfollows. Since the exact details require fairly technical calculations, we leave them to the Journal version.\nTheorem 3:\nThe quantity\n\n(cid:114)\n\n(cid:90)\n\n(cid:90)\n\n\u03b1 :=\n\n\u03c0\nt\n\nS1\n\nS2\n\nKt(x, y)\u03c8t(x)\u03c8t(y)dxdy\n\nis similar to the heat that would \ufb02ow in time t from one part to another if the \ufb01rst were heated proportional to p.\nIntuitively, the heat that would \ufb02ow from one part to the other in a small interval ought to be related to the volume of\nS p(s)ds. To prove this relationship, we bound \u03b1 both\nthe boundary between these two parts, which in our setting is\nabove and below in terms of the weighted volume and condition number of the boundary. These bounds are obtained\n\u03c4 , which is when S is a sphere of radius \u03c4. In\nby making comparisons with the \u201cworst case\u201d, given condition number 1\norder to obtain a lower bound on \u03b1, we observe that if B2 is the nearest ball of radius \u03c4 contained in S1 to a point P\n\nin S2 that is within \u03c4 of S1, (cid:90)\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx \u2265\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx,\n\nS1\n\nB2\n\nas in Figure 4.\nSimilarly, to obtain an upper bound on \u03b1, we observe that if B1 is a ball or radius \u03c4 in S2, tangent to B2 at the point\nof S nearest to P ,\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx \u2264\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx.\n\nBc\n1\n\nWe now indicate how a lower bound is obtained for\n\n(cid:90)\n\nS1\n\n(cid:112)\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx.\n\n(cid:82)\n\n(cid:107)x\u2212P(cid:107)>R\n\nA key observation is that for R =\n\n2dt ln(1/t),\n\nKt(x, P )dx (cid:191) 1. For this reason, only the portions of B2\n\nnear P contribute to the the integral\n\nIt turns out that a good lower bound can be obtained by considering the integral over H2 instead, where H2 is a\nhalfspace whose boundary is at a distance \u03c4 \u2212 R2\n\n2\u03c4 from the center as in \ufb01gure 4.\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx.\n\nB2\n\n6\n\n(cid:90)\n\n(cid:90)\n\nB2\n\n\fFigure 4: The density of heat diffusing to point P from S1 in the left panel is less or equal to the density of heat\ndiffusing to P from B2 in the right panel.\n\nFigure 5: The density of heat received by point P from B2 in the left panel can be approximated by the density of heat\nreceived by P from the halfspace H2 in the right panel.\n\n7\n\n\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001B2\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001PB2B1\u0001\u0001S1P2B\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001\u0001PH2P2R\u03c42\fAn upper bound for\n\n(cid:90)\n\nBc\n1\n\nKt(x, P )\u03c8t(x)\u03c8t(P )dx\n\nis obtained along similar lines.\nThe details of this proof will be presented in the Journal version.\n\n5 Conclusion\n\nIn this paper we take a step towards a probabilistic analysis of graph based methods for clustering. The nodes of the\ngraph are identi\ufb01ed with data points drawn at random from an underlying probability distribution on a continuous\ndomain. For a \ufb01xed partition we show that the cut-size of the graph partition converges to the weighted volume of\nthe boundary separating the two regions of the domain. The rates of this convergence are analyzed. If one is able\nto generalize our result uniformly over all partitions, this allows us to relate ideas around graph based partitioning to\nideas surrounding Low Density Separation. The most important future direction would be to achieve similar results\nuniformly over balanced partitions.\n\nReferences\n[1] M. Belkin and P. Niyogi (2004).\u201cSemi-supervised Learning on Riemannian Manifolds.\u201d In Machine Learning\n\n56, Special Issue on Clustering, 209-239.\n\n[2] M. Belkin and P. Niyogi. \u201cToward a theoretical foundation for Laplacian-based manifold methods.\u201d COLT 2005.\n[3] A. Blum and S. Chawla, \u201cLearning from labeled and unlabeled data using graph mincuts\u201c, ICML 2001.\n[4] O.Chapelle, J. Weston, B. Scholkopf, \u201cCluster kernels for semi-supervised learning\u201d, NIPS 2002.\n[5] O. Chapelle and A. Zien, \u201cSemi-supervised Classi\ufb01cation by Low Density Separation\u201d, AISTATS 2005.\n[6] Chris Ding, Spectral Clustering, ICML 2004 Tutorial.\n[7] Samuel Kutin, Partha Niyogi, \u201cAlmost-everywhere Algorithmic Stability and Generalization Error.\u201d, UAI 2002,\n\n275-282\n\n[8] S. Kutin, TR-2002-04, \u201cExtensions to McDiarmid\u2019s inequality when differences are bounded with high proba-\n\nbility.\u201d Technical report TR-2002-04 at the Department of Computer Science, University of Chicago.\n\n[9] S. Lafon, Diffusion Maps and Geodesic Harmonics, Ph. D. Thesis, Yale University, 2004.\n[10] M. Hein, J.-Y. Audibert, U. von Luxburg, From Graphs to Manifolds \u2013 Weak and Strong Pointwise Consistency\n\nof Graph Laplacians, COLT 2005.\n\n[11] U. von Luxburg, M. Belkin, O. Bousquet, Consistency of Spectral Clustering, Max Planck Institute for Biological\n\nCybernetics Technical Report TR 134, 2004.\n\n[12] M. Meila and J. Shi. A Random Walks View of Spectral Segmentation, NIPS 2001.\n[13] B. Nadler, S. Lafon, R. R. Coifman,and I. G. Kevrekidis. Diffusion Maps, Spectral Clustering and Eigenfunctions\n\nof Fokker-Planck Operators, NIPS 2006.\n\n[14] J. Shi and J. Malik. \u201cNormalized cuts and image segmentation.\u201d\n[15] X. Zhu, J. Lafferty and Z. Ghahramani, Semi-supervised learning using Gaussian \ufb01elds and harmonic functions,\n\nICML 2003.\n\n8\n\n\f", "award": [], "sourceid": 3000, "authors": [{"given_name": "Hariharan", "family_name": "Narayanan", "institution": null}, {"given_name": "Mikhail", "family_name": "Belkin", "institution": null}, {"given_name": "Partha", "family_name": "Niyogi", "institution": null}]}