{"title": "Parametric Embedding for Class Visualization", "book": "Advances in Neural Information Processing Systems", "page_first": 617, "page_last": 624, "abstract": null, "full_text": "Parametric Embedding for Class Visualization\n\nTomoharu Iwata, Kazumi Saito, Naonori Ueda\n\nNTT Communication Science Laboratories\n\nNTT Corporation\n\n2-4 Hikaridai Seika-Cho Soraku-gun Kyoto, 619-0237 JAPAN\n\nfiwata,saito,uedag@cslab.kecl.ntt.co.jp\n\nSean Stromsten, Thomas L. Grif\ufb01ths, Joshua B. Tenenbaum\n\nDepartment of Brain and Cognitive Sciences\n\nMassachusetts Institute of Technology\n\nfsean s,gruffydd,jbtg@mit.edu\n\nAbstract\n\nIn this paper, we propose a new method, Parametric Embedding (PE), for\nvisualizing the posteriors estimated over a mixture model. PE simultane-\nously embeds both objects and their classes in a low-dimensional space.\nPE takes as input a set of class posterior vectors for given data points,\nand tries to preserve the posterior structure in an embedding space by\nminimizing a sum of Kullback-Leibler divergences, under the assump-\ntion that samples are generated by a Gaussian mixture with equal covari-\nances in the embedding space. PE has many potential uses depending\non the source of the input data, providing insight into the classi\ufb01er\u2019s be-\nhavior in supervised, semi-supervised and unsupervised settings. The PE\nalgorithm has a computational advantage over conventional embedding\nmethods based on pairwise object relations since its complexity scales\nwith the product of the number of objects and the number of classes. We\ndemonstrate PE by visualizing supervised categorization of web pages,\nsemi-supervised categorization of digits, and the relations of words and\nlatent topics found by an unsupervised algorithm, Latent Dirichlet Allo-\ncation.\n\n1\n\nIntroduction\n\nRecently there has been great interest in algorithms for constructing low-dimensional\nfeature-space embeddings of high-dimensional data sets. These algorithms seek to cap-\nture some aspect of the data set\u2019s intrinsic structure in a low-dimensional representation\nthat is easier to visualize or more ef\ufb01cient to process by other learning algorithms. Typi-\ncal embedding algorithms take as input a matrix of data coordinates in a high-dimensional\nambient space (e.g., PCA [5]), or a matrix of metric relations between pairs of data points\n(MDS [7], Isomap [6], SNE [4]). The algorithms generally attempt to map all and only\nnearby input points onto nearby points in the output embedding.\n\nHere we consider a different sort of embedding problem with two sets of points X =\n\n\ffx1; : : : ; xN g and C = fc1; : : : ; cK g, which we call \u201cobjects\u201d (X) and \u201cclasses\u201d (C). The\ninput consists of conditional probabilities p(ckjxn) associating each object xn with each\nclass ck. Many kinds of data take this form: for a classi\ufb01cation problem, C may be the set\nof classes, and p(ckjxn) the posterior distribution over these classes for each object xn; in a\nmarketing context, C might be a set of products and p(ckjxn) the probabilistic preferences\nof a consumer; or in language modeling, C might be a set of semantic topics, and p(ckjxn)\nthe distribution over topics for a particular document, as produced by a method like Latent\nDirichlet Allocation (LDA) [1]. Typically, the number of classes is much smaller than the\nnumber of objects, K << N.\n\nWe seek a low-dimensional embedding of both objects and classes such that the distance\nbetween object n and class k is monotonically related to the probability p(ckjxn). This\nembedding simultaneously represents not only the relations between objects and classes,\nbut also the relations within the set of objects and within the set of classes \u2013 each de\ufb01ned in\nterms of relations to points in the other set. That is, objects that tend to be associated with\nthe same classes should be embedded nearby, as should classes that tend to have the same\nobjects associated with them. Our primary goals are visualization and structure discovery,\nso we typically work with two- or three-dimensional embeddings.\n\nObject-class embeddings have many potential uses, depending on the source of the input\ndata. If p(ckjxn) represents the posterior probabilities from a supervised Bayesian clas-\nsi\ufb01er, an object-class embedding provides insight into the behavior of the classi\ufb01er: how\nwell separated the classes are, where the errors cluster, whether there are clusters of objects\nthat \u201cslip through a crack\u201d between two classes, which objects are not well captured by\nany class, and which classes are intrinsically most confusable with each other. Answers to\nthese questions could be useful for improved classi\ufb01er design. The probabilities p(ckjxn)\nmay also be the product of unsupervised or semi-supervised learning, where the classes\nck represent components in a generative mixture model. Then an object-class embedding\nshows how well the intrinsic structure of the objects (and, in a semi-supervised setting, any\ngiven labels) accords with the clustering assumptions of the mixture model.\nOur speci\ufb01c formulation of the embedding problem assumes that each class ck can be\nrepresented by a spherical Gaussian distribution in the embedding space, so that the em-\nbedding as a whole represents a simple Gaussian mixture model for each object xn. We\nseek an embedding that matches the posterior probabilities for each object under this Gaus-\nsian mixture model to the input probabilities p(ckjxn). Minimizing the Kullback-Leibler\n(KL) divergence between these two posterior distributions leads to an ef\ufb01cient algorithm,\nwhich we call Parametric Embedding (PE).\n\nPE can be seen as a generalization of stochastic neighbor embedding (SNE). SNE corre-\nsponds to a special case of PE where the objects and classes are identical sets. In SNE, the\nclass posterior probabilities p(ckjxn) are replaced by the probability p(xmjxn) of object\nxn under a Gaussian distribution centered on xm. When the inputs (posterior probabili-\nties) to PE come from an unsupervised mixture model, PE performs unsupervised dimen-\nsionality reduction just like SNE. However, it has several advantages over SNE and other\nmethods for embedding a single set of data points based on their pairwise relations (e.g.,\nMDS, Isomap). It can be applied in supervised or semi-supervised modes, when class la-\nbels are available. Because its computational complexity scales with N K, the product of\nthe number of objects and the number of classes, it can be applied ef\ufb01ciently to data sets\nwith very many objects (as long as the number of classes remains small). In this sense,\nPE is closely related to landmark MDS (LMDS) [2], if we equate classes with landmarks,\nobjects with data points, and (cid:0) log p(ckjxn) with the squared distances input to LMDS.\nHowever, LMDS lacks a probabilistic semantics and is only suitable for unsupervised set-\ntings. Lastly, even if hard classi\ufb01cations are not available, it is often the relations of the\nobjects to the classes, rather than to each other, that we are interested in.\n\n\fAfter describing the mathematical formulation and optimization procedures used in PE\n(Section 2), we present applications to visualizing the structure of several kinds of class\nposteriors. In section 3, we look at supervised classi\ufb01ers of hand-labeled web pages. In\nsection 4, we examine semi-supervised classi\ufb01ers of handwritten digits. Lastly, in section 5,\nwe apply PE to an unsupervised probabilistic topics model, treating latent topics as classes,\nand words as objects. PE handles these datasets easily, in the last producing an embedding\nfor over 26,000 objects in a little over a minute (on a 2GHz Pentium computer).\n\n2 Parametric Embedding method\n\nGiven as input conditional probabilities p(ckjxn), PE seeks an embedding of objects with\ncoordinates rn and classes with coordinates (cid:30)k, such that p(ckjxn) is approximated as\nclosely as possible by the posterior probabilities from a unit-variance spherical Gaussian\nmixture model in the embedding space:\n\np(ckjrn) =\n\np(ck) exp((cid:0) 1\nl=1 p(cl) exp((cid:0) 1\n\n2 k rn (cid:0) (cid:30)k k2)\n\n2 k rn (cid:0) (cid:30)l k2)\n\n:\n\n(1)\n\nPK\n\nHere k (cid:1) k is the Euclidean norm in the embedding space. When the conditional probabil-\nities p(ckjxn) arise as posterior probabilities from a mixture model, we will also typically\nbe given priors p(ck) as input; otherwise the p(ck) terms above may be assumed equal.\nto measure the degree of correspondence between input probabilities\nIt\nand embedding-space probabilities using a sum of KL divergences for each object:\nn=1 KL(p(ckjxn)jjp(ckjrn)). Minimizing this sum w.r.t. fp(ckjrn))g is equivalent to\n\nis natural\n\nminimizing the objective function\n\nPN\n\nE(frng; f(cid:30)kg) = (cid:0)\n\np(ckjxn) log p(ckjrn):\n\n(2)\n\nN\n\nK\n\nXn=1\n\nXk=1\n\nSince this minimization problem cannot be solved analytically, we employ a coordinate\ndescent method. We initialize f(cid:30)kg, and we iteratively minimize E w.r.t. to f(cid:30)kg or frng\nwhile \ufb01xing the other set of parameters, until E converges.\n\nDerivatives of E are:\n\n@E\n@ rn\n\n=\n\nK\n\nXk=1\n\n(cid:11)n;k(rn (cid:0) (cid:30)k) and\n\n@E\n@(cid:30)k\n\n=\n\nN\n\nXn=1\n\n(cid:11)n;k((cid:30)k (cid:0) rn);\n\n(3)\n\nwhere (cid:11)n;k = p(ckjxn) (cid:0) p(ckjrn). These learning rules have an intuitive interpretation\n(analogous to those in SNE) as a sum of forces pulling or pushing rn ((cid:30)k) depending on the\nsign of (cid:11)n;k. Importantly, the Hessian of E w.r.t. frng is a semi-positive de\ufb01nite matrix:\n\n@ 2E\n@ rn@ r\n\n0\nn\n\n=\n\nK\n\nXk=1\n\np(ckjrn)(cid:30)k(cid:30)0\n\nk (cid:0) K\nXk=1\n\np(ckjrn)(cid:30)k! K\nXk=1\n\n0\n\np(ckjrn)(cid:30)k!\n\n(4)\n\nsince the r.h.s. of (4) is exactly a covariance matrix. Thus we can \ufb01nd the globally optimal\nsolution for frng given f(cid:30)kg.1 The computational complexity of PE is O(N K), which\nis much more ef\ufb01cient than that of pairwise (dis)similarity-based methods with O(N 2)\ncomputations (such as SNE, MDS, or Isomap).\n\n1In our experiments, we found that optimization proceeded more smoothly with a regularized\n\nobjective function, J = E + (cid:17)rPN\n\nn=1 k rn k2 +(cid:17)(cid:30)PK\n\nk=1 k (cid:30)k k2, where (cid:17)r; (cid:17)(cid:30) > 0.\n\n\f3 Analyzing supervised classi\ufb01ers on web data\n\nIn this section, we show how PE can be used to visualize the structure of labeled data\n(web pages) in a supervised classi\ufb01cation task. We also compare PE with two conventional\nmethods, MDS [7] and Fisher linear discriminant analysis (FLDA) [3]. MDS seeks a low-\ndimensional embedding that preserves the input distances between objects.\nIt does not\nnormally use class labels for data points, although below we discuss a way to apply MDS\nto label probabilities that arise in classi\ufb01cation. FLDA, in contrast, naturally uses labeled\ndata in constructing a low-dimensional embedding. It seeks a a linear projection of the\nobjects\u2019 coordinates in a high-dimensional ambient space that maximizes between-class\nvariance and minimizes within-class variance.\n\nThe set of objects comprised 5500 human-classi\ufb01ed web pages: 500 pages sampled from\neach of 11 top level classes in Japanese directories of Open Directory (http://dmoz.org/).\nPages with less than 50 words, or which occurred under multiple categories, were elim-\ninated. A Naive Bayes (NB) classi\ufb01er was trained on the full data (represented as word\nfrequency vectors). Posterior probabilities p(ckjxn) were calculated for classifying each\nobject (web page), assuming its true class label was unknown. These probabilities, as well\nas estimated priors p(ck), form the input to PE.\nFig.1(a) shows the output of PE, which captures many features of this data set and classi\ufb01-\ncation algorithm. Pages belonging to the same class tend to cluster well in the embedding,\nwhich makes sense given the large sample of labeled data. Related categories are located\nnearby: e.g., sports and health, or computers and online-shopping. Well-separated clusters\ncorrespond to classes (e.g. sports) that are easily distinguished from others. Conversely,\nregional pages are dispersed, indicating that they are not easily classi\ufb01ed. Distinctive pages\nare evident as well: a few pages that are scattered among the objects of another category\nmight be misclassi\ufb01ed. Pages located between clusters are likely to be categorized in multi-\nple classes; arcs between two classes show subsets of objects that distribute their probability\namong those two classes and no others.\n\nFig.1(b) shows the result of MDS applied to cosine distances between web pages. No\nlabeled information is used (only word frequency vectors for the pages), and consequently\nno class structure is visible. Fig.1(c) shows the result of FLDA. To stabilize the calculation,\nFLDA was applied only after word frequencies were smoothed via SVD. FLDA uses label\ninformation, and clusters together the objects in each class better than MDS does. However,\nmost clusters are highly overlapping, and the separation of classes is much poorer than\nwith PE. This seems to be a consequence of FLDA\u2019s restriction to purely linear projections,\nwhich cannot, in general, separate all of the classes.\n\nFig.1(d) shows another way of embedding the data using MDS, but this time applied to\nEuclidean distances in the (K (cid:0) 1)(cid:0)dimensional space of posterior distributions p(ckjxn).\nPages belonging to the same class are de\ufb01nitely more clustered in this mode, but still the\nclusters are highly overlapping and provide little insight into the classi\ufb01er\u2019s behavior. This\nversion of MDS uses the same inputs as PE, rather than any high-dimensional word fre-\nquency vectors, but its computations are not explicitly probabilistic. The superior results of\nPE (Fig.1(a)) illustrate the advantage of optimizing an appropriate probabilistic objective\nfunction.\n\n4 Application to semi-supervised classi\ufb01cation\n\nThe utility of PE for analyzing classi\ufb01er performance may best be illustrated in a semi-\nsupervised setting, with a large unlabeled set of objects and a smaller set of labeled objects.\nWe \ufb01t a probabilistic classi\ufb01er based on the labeled objects, and we would like to visualize\nthe behavior of the classi\ufb01er applied to the unlabeled objects, in a way that suggests how\n\n\f(a) PE\n\n(b) MDS (word frequencies)\n\n(c) FLDA\n\n(d) MDS (posteriors)\n\nFigure 1: The visualizations of categorized web pages. Each of the 5500 web pages is show\nby a particle with shape indicating the page\u2019s class.\n\naccurate the classi\ufb01er is likely to be and what kinds of errors it is likely to make.\n\nWe constructed a simple probabilistic classi\ufb01er for 2558 handwritten digits (classes 0-4)\nfrom the MNIST database. The classi\ufb01er was based on a mixture model for the density\nof each class, de\ufb01ned by selecting either 10 or 100 digits uniformly at random from each\nclass and centering a \ufb01xed-covariance Gaussian (in pixel space) on each of these examples\n\u2013 essentially a soft nearest-neighbor method. The posterior distribution over this classi\ufb01er\nfor all 2558 digits was submitted as input to PE.\n\nThe resulting embeddings allow us to predict the classi\ufb01ers\u2019 patterns of confusions, calcu-\nlated based on the true labels for all 2558 objects. Fig. 2 shows embeddings for both 10\nlabels/class and 100 labels/class. In both cases we see \ufb01ve clouds of points corresponding\nto the \ufb01ve classes. The clouds are elongated and oriented roughly towards a common cen-\nter, forming a star shape (also seen to some extent in our other applications). Objects that\nconcentrate their probability on only one class will lie as far from the center of the plot as\npossible \u2013 ideally, even farther than the mean of their class, because this maximizes their\n\n\fEstimated class\n\n0\n\n1\n\n2\n\n3\n\n4\n\nxx x\nx\n.\n.\nxxxxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n404\n\nl\n\n.\n\n.\n\n.\n\n.\n\n3\n\n0\n\n0\n\n1\n\n2\n\n2\n\n1\n\n2\n\n1\n\n6\n\n0\n\n5\n\n4\n\n5\n\n.\n.\n\n36\n\n26\n\n49\n\n26\n\n17\n\n.\n.\n.\n\n330\n\n294\n\n278\n\n117\n\n144\n\n134\n\n557\n\n117\n\n3\n.\n4\n\ns\ns\na\nc\n \ne\nu\nr\nT\n\nxxxxxxxxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n0\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n4\n.\n.\n.\n.\n.\n.\n.\nxxxxxx\n.\nxxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n3\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n1\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n2\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n... .\n.\n.\n.\n.\n.\n.\n.\n...\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxxxxxxxxx\n.\n.\n.\n..\n...\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n.\n.\n\n.\n.\n\n.\n\n.\n\n.\n\n.\n\n.\n\n.\n\n.\n\n.\n.\n.\n.\n.\n.\nxxx xxxxxxx\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\nEstimated class\n\n4\n\n0\n\n1\n\n2\n\n0\n\n486\n\nl\n\n.\n\n.\n\n.\n\n1\n\n2\n\n0\n\n0\n\n2\n\n1\n\n0\n\n8\n\n1\n\n8\n\n1\n\n2\n\n3\n\n2\n\n3\n\n0\n\n0\n\n1\n\n2\n\n0\n\n4\n\n.\n.\n\n.\n.\n\n16\n\n47\n\n34\n\n74\n\n455\n\n388\n\n559\n\n471\n\ne\nu\nr\nT\n\n.\n.\n.\n.\n.\n.\n.\n\ns\ns\na\nc\n \n\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\nxx\nx\nx\nx\nxx\nx\nxxxxxxx\nx\n.\n.\n.\n..\nx\nx\n.\n.\nxx\nx\n.\n.\nx\nx\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx\n.\n.\nxxx\n.\n.\nxx\n.\n.\nx\n.\nx\nxxxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\nx\nxxx\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nxx\n.\n.\nx\nx\n.\nx\nx\n.\n.\n.\n.\n.\nxx\n.\nx\nx\n.\n.\n.\n.\n.\n.\nxx\n.\nx\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\nx\n.\nx\nx\n.\n.\n.\nx\n.\nx\n.\nx\n.\nx\nx\n.\nx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\nx\n.\n.\nxx\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\nx\nx\n.\n2\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\nxxx\nx\nx\nx\nxx\n.\nxx\nx\nxx\nxx\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n..\nx\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\nxxx\n.\n.\n.\nx\n.\n.\n.\nxx\n.\nxxx\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx x\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nxx\nx\n.\n.\nx x\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\nx\nx\nx\nxx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n4\n.\n.\n.\nx\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx x\nx\nx\n.\n.\nx\nx\n.\n. .\n.\nxx\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n..\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\nx\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\nx\n.\n.\n. .\n.\n.\nxx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\nxxx\nx xx\nxx\n.\n.\nxx\nxx\nx\n.\n.\n.\nxx\n.\n.\n.\nxx\n.\n.\n.\n.\n..\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx x\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx\nx\n.\n.\nx\n.\nx\n.\nxx\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx x\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx x\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx x\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n3\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\nx x\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx x\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\nx\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxxx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n1\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\nx\n.\n.\n.\n.\nx\n.\n..\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxxx\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\nxxxxxxxxx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx\n.\n..\n.\n.\n.\nxx\nxxxxxx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\nxxxxxx\n.\n.\n.\n.\n.\n.\n.\n.\nxxx\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\nxxx\n.\n.\nxx\n.\n.\n.\n..\n.\n.\n.\n.\n.\n. ..\n....\n.\n.\n..\n.\n.\n..\n.\n.\nx\nxxx\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n..\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\nxxx\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\nx\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\nxxxx\n..\n.\n.\n.\n..\n.\n..\n..\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n..\n.\nx\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\nxxxxxxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\nx\n.\n..\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\nx\n.\n.\n.\n..\n.\n.\nx\n.\n.\nx\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n0\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx x\n.\n.\n.\n.\n.\n.\n.\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\nx\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\nx\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nxx\n.\n.\nxx\n.\n.\n.\n.\nx\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\nx\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx\nx\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\nx\n.\n.\nxx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\n.\nx\n.\n.\nx\n.\nx\nx\nx\n.\n.\n.\n..\n.\nx\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx\n.\n.\nx\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\nx\n.\n.\n.\n.\n.\nx\n.\nx\n.\n.\nxxx\nx\nx\n.\n.\n.\n.\n.\n.\nx\n.\n. .\n.\nxx\nx\nxx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxx\n.\n.\nxxx\n.\nx\nx\nx\n.\n.\n.\nx\n.\n.\n.\nxx\nxx\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nxxx\nxx\nx\n.\n.\nx\n.\n.\n.\n.\n.\nx\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n.\nx\n.\n.\n.\nxx\n.\nx\nx\n.\n.\n.\n.\n.\n.\n.\n.\n\n.\n\n.\n\n.\n\n.\n\n(a) PE with 10 labels/class\n\n(b) PE with 100 labels/class\n\nFigure 2: Parametric embeddings for handwritten digit classi\ufb01cation. Each dot represents\nthe coordinates rn of one image. Boxed numbers represent the class means (cid:30)k. (cid:2)\u2019s show\nlabeled examples used to train the classi\ufb01er. Images of several unlabeled digits are shown\nfor each class.\n\nposterior probability on that class. Moving towards the center of the plot, objects become\nincreasingly confused with other classes.\n\nRelative to using only 10 labels/class, using 100 labels yields clusters that are more dis-\ntinct, re\ufb02ecting better between-class discrimination. Also, the labeled examples are more\nevenly spread through each cluster, re\ufb02ecting more faithful within-class models and less\nover\ufb01tting. In both cases, the \u20181\u2019 class is much closer than any other to the center of the\nplot, re\ufb02ecting the fact that instances of other classes tend to be mistaken for \u20181\u2019s. Instances\nof other classes near the \u20181\u2019 center also tend to look rather \u201cone-like\u201d \u2013 thinner and more\nelongated. The dense cluster of points just outside the mean for \u20181\u2019 re\ufb02ects the fact that \u20181\u2019s\nare rarely mistaken for other digits. In Fig. 2(a), the \u20180\u2019 and \u20183\u2019 distributions are particu-\nlarly overlapping, re\ufb02ecting that those two digits are most readily confused with each other\n(apart from 1). The \u2018webbing\u2019 between the diffuse \u20182\u2019 arm and the tighter \u20183\u2019 arm re\ufb02ects\nthe large number of \u20182\u2019s taken for \u20183\u2019s. In Fig. 2(b), that \u2018webbing\u2019 persists, consistent with\nthe observation that (again, apart from many mistaken responses of 1) the confusion of \u20182\u2019s\nfor \u20183\u2019s is the only large-scale error these larger data permit.\n\n5 Application to unsupervised latent class models\n\nIn the applications above, PE was applied to visualize the structure of classes based at least\nto some degree on labeled examples. The algorithm can also be used in a completely un-\nsupervised setting, to visualize the structure of a probabilistic generative model based on\nlatent classes. Here we illustrate this application of PE by visualizing a semantic space of\nword meanings: objects correspond to words, and classes correspond to topics in a latent\nDirichlet allocation (LDA) model [1] \ufb01t to a large (>37,000 documents, >12,000,000 word\ntokens) corpus of educational materials for \ufb01rst grade to college (TASA). The problem of\nmapping a large vocabulary is particularly challenging, and, with over 26,000 objects (word\ntypes), prohibitively expensive for pairwise methods. Again, PE solves for the con\ufb01gura-\ntion shown in about a minute.\n\nIn LDA (not to be confused with FLDA above), each topic de\ufb01nes a probability distribution\n\n\fIMPLEMENTED\n\nCARPENTRY\n\nSTATEWIDE\n\nAUTISTIC\n\nGROUP\n\nDIFFICULTY\n\nAPPARATUS\n\nCHEMISTRY\n\nFRACTIONAL\n\nPHENOMENON\n\nDETECTION\n\n.\n.\n\n.\n\nCONTENT\n\nSCHOLARSHIP\n\nADMINISTRATION\n\n.\n\nPHASE\n\nCOVALENTLY\n\nALKENE\n\nADSORPTION ACTIVATED\n\n.\n.\n.\n.\n.\n\n.\n.\n.\n\n..\n.\n.\n.\n.\n.\n\nSTRATIFIED\nAREAS\n\n.\n\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\n...\n.\n.\n.....\n....\n...\n..\n.\n..\n...\n.\n.\n..\n.\n.....\n..\n.\n...\n.\n.\n..\n...\n...\n.\n..\n..\n.\n.\n.\n...\n....\n.\n.\n.\n.\n..\n..\n.\n..\n...\n.\n.....\n.\n....\n..\n.\n.\n...\n..\n.\n.\n..\n.\n.\n..\n.\n.\n.\n....\n.\n.\n..\n.....\n.\n.\n....\n.\n.\n..\n..\n.\n.\n..\n.\n......\n.\n.\n.....\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n...\n.\n.\n..\n.\n.\n...\n..\n.\n.\n.\n..\n.\n...\n..\n.\n..\n..\n.\n.\n..\n.\n.\n..\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n..\n.\n.\n....\n.\n..\n.\n.\n.\n.......\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n...\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.....\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n..\n..\n..\n.\n.\n.\n.\n.\n...\n...\n...\n....\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n..\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n..\n.\n..\n.\n.\n....\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n....\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n....\n.\n.\n.\n.\n..\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n..\n.\n.\n.\n.\n.\n..\n..\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n..\n.\n..\n...\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n....\n..\n.\n..\n....\n.\n.\n.\n.\n.....\n.\n..\n.\n.\n.\n..\n..\n.....\n.\n..\n.\n.\n.\n.\n..\n.\n..\n.\n..\n..\n.\n..\n.\n.\n.\n.\n....\n...\n..\n.\n..\n.....\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n...\n.\n.\n.\n..\n.\n...\n.\n...\n.\n.\n.\n.\n..\n.\n..\n.\n..\n...\n....\n.\n.\n.\n..\n.\n.\n.\n..\n.\n..\n.\n..\n.\n...\n. .\n..\n.\n.\n.\n...\n.\n..\n.\n.\n..\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n....\n..\n..\n.\n.\n....\n.\n.\n...\n.\n.\n.\n....\n...\n.\n..\n...\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.....\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n...\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n..\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n..\n.\n..\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n..\n.\n..\n.\n.\n..\n.\n.\n....\n..\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n...\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\nCOMPOSITION DISSOLVES\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nMIXTURE\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nchemistry\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\ntraining/education\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nEFFORTS CONCEPTS\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\nPATIENCE\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.............................\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...........\n.\n.\n......... ..\n.\n.\n.\n.\n.\n...........\n.\n.\n.\n.\n.\n..\n.....\n.\n.\n.\n.\n.\n.\n.....\n.\n...............\n.........\n....\n...\n..\n................\n................\n...............\n................\n....................................\n............................\n.....................\n.......\n.....\n..\n.\n....\n.\n.....\n......\n.........\n..................\n....\n....................\n........\n..........\n...............\n..............\n.......\n....................................\n....................\n................\n...........\n.............\n.......................\n.........................\n..................\n...................\n...\n....\n.......\n....\n................\n.........................\n....\n....\n..................\n...............\n........\n.\n.....\n......\n.......\n......\n.................\n.................\n..\n.....\n...........\n.........\n.............\n.....\n........\n....\n.............\n.....\n...........\n..........\n....\n..............\n...............\n.\n...............\n...\n................\n.......\n...................................\n..........\n............\n...........\n......\n....\n......\n..........\n................\n.....\n.................\n...........\n.........\n........\n................\n...\n.............\n..\n....\n....\n..............\n........\n..........\n...................\n...........\n.............\n..............\n..............\n....\n...\n......\n......\n...............\n.......\n..............\n.....\n............\n..............\n........\n........................\n.........\n......................\n.....\n...................................\n...................\n.......................\n...........\n.........\n.........\n...............\n...........\n.......\n.\n.\n.......\n.......\n.........\n.............\n......\n.....\n.......\n..........\n.......\n.....\n.......\n.........\n.......\n..\n.\n.........\n..\n...................\n....................\n...\n......................................\n..........\n......\n..............\n....\n.....................\n..............\n...\n...........\n.....\n.........\n.................\n.........\n..\n..\n......\n............\n...\n..\n......\n.......\n..................\n......\n......\n......\n..\n.....................\n............\n...\n.........\n.....\n....\n.......\n.........\n....\n.................\n.........\n..............\n.................\n.\n............\n.............\n...\n..\n.....\n...........\n......................\n............\n...........\n...\n.\n.................\n.\n....\n.............\n........\n...\n..........\n.....\n.......\n....\n...........\n.......\n.......................\n...\n..\n.............\n......\n......\n............\n......\n............\n..........\n.....\n.................................\n.....\n...\n..........\n...........\n................\n.......\n...................\n..\n.\n...\n...\n...\n.......\n.\n.......\n.\n....\n............................\n.\n.................\n...................\n....\n..\n....\n.\n.........\n.................\n...\n...........\n........\n......\n........\n.......\n.........\n.................\n.........\n.....\n.........\n..............................\n...........\n..................\n.......\n...\n.....\n.............\n......................\n........................\n......\n..............\n...................\n...\n..........\n...\n.....\n.......\n........\n.\n.....\n....\n.......\n......\n........\n...\n.\n.............\n.......\n...........\n...........\n.........\n.\n..\n...........\n........\n.\n..............\n.\n...........\n................\n......\n..\n...\n............\n.................\n.....\n.................................\n...\n....\n..............\n..................\n....\n...\n....................................\n.............\n.............\n..........\n...............\n.\n.....................\n..\n.\n.................\n......\n.........................\n....\n........\n............\n........\n.................\n.....\n...............\n.............\n....\n...\n......\n...................\n.............\n...\n...........\n.........\n...............\n...........\n........\n..............\n...............................\n..............\n......\n...........\n....\n.............\n.........................\n...............\n........\n.\n....\n..........\n...............\n.......\n....\n........\n.....................................\n..............\n..........................\n..........................\n..\n................\n....................\n....\n...............\n......\n.......................\n.......\n.........................\n......\n..........\n.\n...\n.............\n.........\n.....\n............\n..\n..............\n........................................\n..............\n........................\n...................\n..\n..........\n.\n........................\n...................\n..\n.....\n..\n....\n..........\n.....\n..\n....\n......\n...\n...\n...\n....................\n..................\n.....\n...........\n.................\n................\n...................\n.....\n..\n..................\n...\n........\n...........................\n.............\n.............\n..................\n..........\n....\n....\n..........\n..\n.................\n...............\n.................\n..\n...........\n.......\n..\n..\n...\n..................\n.\n........\n.............\n.....\n..\n..............\n............\n....\n.....\n....\n.......\n...\n...\n....\n....\n..\n.....\n...\n.........\n...\n..............\n..\n...\n...\n...\n......\n....................\n...\n.....\n....................\n............\n...\n.\n.......\n...\n..\n......\n...............\n..........................\n...........\n.......\n..............\n......\n......\n.\n....................\n........................\n.\n...\n.................\n.......\n...................\n................\n.....\n........\n.....\n.................\n......\n....\n..\n........\n..\n........\n.....\n.....\n.......\n.......\n...\n..\n....\n.........\n.....................\n.............\n.....\n.......\n............\n....\n...................\n.....\n...\n....\n...\n..\n...\n....\n.........\n.......\n..\n.\n.............\n..............\n...\n......\n.........\n..........\n.\n..............\n...........\n................\n......\n....\n.........\n........\n.\n...\n.\n.......\n.\n..\n.....\n...\n...\n.\n..............\n..................\n............\n....\n...\n..............\n...............\n..........\n...................\n....\n.......\n........\n....\n.........\n...............\n..\n........\n...\n......\n.........\n........\n..........................\n....\n.........\n............\n.............\n.....\n.\n.............\n.............\n..\n..\n...\n.......\n..\n....\n.......\n.......\n..\n.....\n.\n...\n........\n.\n........\n.............\n..\n...................\n....\n...\n........................\n....................\n.........\n.........\n.........\n.....\n........\n.......\n.............\n.......\n....\n................\n....\n....\n.....\n.........\n.........\n.......\n........\n.\n....\n..\n..............\n.......\n.............\n...\n......\n.....\n...........................................\n....\n..\n........\n....\n.....\n......\n......\n.........\n.\n...............\n................\n.............\n..........\n...\n......\n.....................\n..\n.........\n........\n........................\n.....\n...........................\n.\n.................\n.........\n.....\n..........\n.....\n....\n......\n......\n......\n.....\n........\n.........\n.........\n.\n..\n.........\n...........\n..........\n...\n...\n..........\n..\n.......................\n..........\n..\n......\n.......\n................\n............\n...\n.\n...\n.....\n.......\n............\n.\n.......\n...\n................\n.\n.........\n......................\n.................\n.......\n.......\n....\n........\n..........\n...........\n.....................\n............\n.......\n.........\n.....\n........\n......................\n....\n.\n.\n........\n..\n...............\n....................\n..........\n........\n..........\n.............................................\n..\n........\n......\n..........\n.....\n..........\n.............\n....\n..............\n......\n....\n..\n....\n.....\n........\n...\n.....\n.........\n.\n...............\n.....\n.......\n...\n.....\n.\n...............\n...\n......\n....\n....\n....\n.\n.........\n...\n..\n.......\n...\n.\n...\n......\n....\n.\n........\n...............\n....\n.....\n.......\n...\n......\n..\n.\n...........\n...............\n........\n........\n..\n..........\n..\n......\n....\n.....\n..\n......\n.\n.........\n...\n........\n..........\n....\n......\n..........\n........\n..........\n.......................\n........\n..............\n.................\n. ..\n............\n.....\n.....\n............\n.................\n...........\n...\n......................\n.........\n..\n..........\n.......\n...\n.........\n....\n..\n.\n..........\n..\n............\n.\n..\n...\n..........\n..........\n.......\n......\n...\n............................\n..\n...\n..............\n.......\n......\n..............\n........\n......\n.....\n...\n....\n.....\n.\n.\n...........\n.....\n..........\n........\n..........\n.......\n...\n.......\n....\n.....\n..........\n................\n...\n.......\n.\n.\n.............\n...\n...\n.......\n..\n.....................\n......\n...........\n..............................\n.......\n................\n........\n.....\n....\n..\n.............\n.....\n...\n............\n.........\n.......\n.\n.................\n...........\n.......\n.............\n....\n........\n.................\n...\n....\n...........\n..\n..\n...........\n.\n...\n.....\n.\n.....\n......\n............\n...............\n......\n....\n.........\n.............\n...........\n.\n...........\n..\n...\n..\n.\n.......\n....\n........................\n.....\n....................\n.....\n......\n...\n.........\n...........\n......\n....................\n.........\n.\n....................\n................\n...\n..........\n...\n...\n...........\n..\n.........\n.\n........\n..\n...........\n................\n....\n....\n..........\n...\n...........\n...\n.......\n................\n.......\n.......\n...\n.....\n.\n....\n......\n.................\n.............\n................\n..........\n......................\n...................\n.\n.\n....\n............\n.\n...\n.......\n.......\n......\n..\n.............\n...\n....\n........\n....\n.........\n.................\n..\n....\n..........\n.\n...\n............\n...................\n....\n...........\n..\n...............\n...........\n........\n......\n......\n......\n.\n.....\n.....\n..............\n..\n....................\n........\n..\n.................\n.........\n......................\n.\n.....\n......\n...\n.\n...........\n..\n..........\n..............\n.\n...........................\n.\n..\n.........\n....\n..\n..............\n...........\n....\n.....\n...........\n......\n.\n..\n...\n....\n..\n...\n...........\n.\n..\n.\n...\n...........\n...\n........\n.......\n....................\n....\n................\n.......\n.....\n............\n.....\n..........................\n.............\n........\n..\n.\n.\n........................\n.......\n......\n......\n..\n..\n......\n..............\n...\n..\n...................\n......\n....\n.......\n....\n........\n.......\n.\n.\n...\n.......\n...............\n...\n..............\n.....\n.......\n..\n.\n..\n......\n...........\n...........\n...\n..\n...\n......\n.......\n...................\n.............\n..........\n...\n......................................\n...\n........\n.\n...\n.......................\n..\n..........................\n......\n.......\n....\n.\n...................\n......\n.....................\n...\n................\n.......\n..\n.........\n.............\n...\n...\n...........\n......................\n...\n....\n...\n............\n...\n...\n.............\n..\n...................\n........\n..........\n..........\n.........\n................\n.\n......\n...............\n..................\n........\n.\n......\n..........\n..\n.........\n......\n....\n....................\n...\n..\n.................\n..................\n....\n....\n......\n..\n.....\n.............\n...\n.......\n.\n........\n..\n......\n....\n...........\n........\n...\n.\n......\n.......\n.\n........\n..\n......\n.......\n...........\n...........\n.........\n.........\n..\n........\n....\n........\n..\n...\n.......\n.......\n.........\n....\n.\n........\n.....\n...........\n...\n..\n............\n........\n.........\n..........\n.....\n....\n..\n..........\n..\n.........\n...............\n...........\n.\n.............\n.......\n.\n.......\n......\n.........\n...\n..........\n......\n..........\n.......\n.\n.............\n..\n.....\n...\n.........\n...\n.........\n........\n..\n.\n..........\n...........\n................\n.........\n...\n.....\n.\n....\n..\n.......................\n..\n....\n.....\n...\n.........\n..\n.....\n.......\n.\n..\n................\n.........\n.....\n..\n....\n.........\n...........\n..\n.......\n...........\n..................\n...........\n......................\n.........\n..................\n.....\n............\n.......\n.................\n........\n.......\n.....\n.......\n......\n.........\n.....\n................\n.........\n.\n.......\n.\n....\n.\n....\n.\n.\n.........\n............\n.....\n..............\n....\n...........\n.\n........\n.....\n..........\n......\n...\n.........\n.\n.....\n......\n.......................\n.....\n...\n......................\n...\n....\n.....\n...\n......\n..\n............\n..........\n....\n...\n.....\n.........\n........\n..........\n...................\n.......\n..............\n....\n......\n.\n.....\n.\n.\n....\n.......\n........\n....\n.........\n......\n........\n..\n.........\n....................\n..\n......\n...\n....\n..........\n........\n....\n..\n...\n..\n....................\n.....\n.........\n.......\n...\n..\n..\n.\n..............\n..\n....\n...\n...\n........\n.....\n.....\n....\n.............\n...............\n...\n.\n.....\n...\n.\n....\n....\n....\n......\n........\n.\n...\n.........................\n...\n..\n...........\n...........\n.........\n........\n...\n..\n...\n......\n....\n........\n.........\n...........\n..\n....\n...\n.........\n...........................\n.....\n........\n...........\n...\n..............\n....\n..........\n......\n........\n.....\n..\n...........\n..\n....\n.\n....\n.....\n.....\n....\n....\n...\n.\n.....\n.........................\n..\n..\n..\n...\n....\n.\n.....\n.\n........\n.\n....\n.......\n......\n..\n.......\n...\n......\n....\n.\n..\n...\n....\n...\n..\n..\n....\n..\n...\n...............\n............\n....\n.........\n..\n.........\n.......\n...\n.\n...\n...\n....\n.\n..........\n...........\n...............\n...\n.........\n.\n........\n........\n...................\n..........\n..\n....\n...............\n.........\n......\n........\n....\n..\n.................\n.......\n.........\n.......\n.\n......\n..\n....\n...\n.............\n.......\n......\n...\n..\n....\n.......\n....\n.......\n.\n...........\n....\n.\n.\n.....\n...\n...................\n.\n.............\n.\n...\n...........\n...\n........\n................\n.......\n.\n.\n.....\n.\n....\n.....\n........\n...\n.......\n.\n....\n......\n.\n.\n................\n...........\n.......\n.....\n..\n...\n..\n..............\n..........\n...........\n..........\n...............\n...\n..........\n......\n........\n....\n....\n..\n.......\n......\n.\n....\n........\n.\n......\n............\n.\n.......\n....\n.......\n..\n.........\n...........\n..............\n..........\n.....\n.\n...\n......\n.....\n......\n......\n...\n.......\n...............\n........\n.....\n...\n.\n.........\n....\n...\n..............................\n.............\n...........\n.......\n...\n......\n....\n......\n..\n....\n..........\n..........\n...........\n.\n........\n.\n.\n........\n......\n....\n......\n...\n............\n...\n..\n...\n..\n..\n..\n...........\n........\n..\n.\n.........\n...\n.................\n......\n....\n..\n.....\n....\n....\n..\n....\n..................\n...........\n...\n..........\n...\n..\n......\n...\n....\n..........\n..\n.............\n.......\n........\n.........\n......\n............\n.\n......\n.......\n.........\n..\n..........\n....\n.........\n......\n.......\n..\n...........\n.........\n..........\n..\n.........\n......\n....\n...\n....\n.\n......\n...\n....\n.....\n..\n..\n......\n.\n........\n.\n.....\n.\n...\n...\n..........\n......\n.\n..\n......\n.........\n..\n.................\n........\n...\n.........\n...\n.....\n.....\n.\n....\n..\n...\n......\n......\n.\n..........\n........\n.......\n......\n..\n.......................\n.\n.........\n.....\n.............\n................\n.....\n.........\n..\n..\n...........\n..........\n....\n......\n.\n.....\n.....\n..\n...\n..\n.\n....\n.\n........\n.....\n....\n.\n.....\n........\n...\n.\n..\n........\n....\n............\n......\n.\n.....\n...\n.....\n...\n....\n..\n.....\n..\n......\n......\n........\n.\n.....\n...\n.\n...\n..\n...\n....\n..............\n.............\n......\n.\n.....\n......\n........\n....\n..........\n.....\n....\n..\n...\n......\n.\n.....\n.....\n.....\n..\n.......\n........\n......\n...\n...\n..\n.....\n...............\n....\n.\n.....\n......\n..........\n.....\n.........\n........\n.........\n....\n....\n......\n.............\n......\n.........\n.....\n.\n.....\n....\n.....\n..\n.....\n....\n....\n.......\n..\n.......\n........\n...\n..........\n.......\n.....\n.........\n........\n.\n..\n..\n...\n....\n.\n.\n........\n...\n.\n..\n....\n....\n.......\n......\n.......\n....\n........\n.\n..\n......\n...\n......\n.....\n...\n...\n....\n.\n.....\n.........\n.\n.\n.\n.........\n..\n...\n...\n....\n...\n....\n........\n...............\n......\n........................\n.....\n..\n..\n....\n.\n..\n..\n.....\n.........\n.....\n..........\n.........\n..\n...\n..\n....\n.....\n.....\n.........\n.............\n..\n...\n...........\n...\n......\n....\n..\n.\n.......\n.......\n..\n.....\n..\n............\n........\n............\n.............\n...\n.\n.\n...\n........\n...\n.....\n.................\n...\n.\n..\n....\n....\n..\n.............\n.....\n.....\n....\n.\n........\n..\n...\n......\n...\n.....\n. ...\n........\n..\n....\n...\n....\n.\n....\n.......\n........\n..\n.......\n.\n..\n....\n...\n....\n....\n...............\n...\n........\n..\n....\n....\n..........\n..\n.\n.....\n...\n..\n...\n.......\n. ..\n.\n.\n...\n.\n......\n.\n......\n..\n..\n...\n....\n...\n..\n.........\n......\n.....\n.\n.\n.........\n.....\n...\n......\n.......\n......................\n.\n.\n......\n....\n...\n...\n..\n..\n...\n..\n.\n...........\n.\n.\n..........\n........\n..\n..\n..\n..\n.\n...\n.............\n...\n.\n..\n..\n.........\n.....\n....\n...\n.....\n.......\n...\n...\n..\n..\n..........\n...\n...\n.....\n..\n.\n................\n......\n.......\n..\n..\n........\n.\n....\n....\n.\n...\n........\n.\n.......\n.\n.\n...\n.....\n.....\n......\n..\n.......\n...\n..\n......\n............\n......\n....\n....\n..\n.\n........\n..........\n.............\n..........\n.........\n..\n......\n....\n.......\n..........\n.......\n...\n..........\n.....\n......\n..\n....\n..\n.\n......\n..\n........\n.\n.............\n........\n.\n......\n..........\n.\n.\n...\n..\n.\n...\n..\n.\n.\n.\n....\n........\n.\n........\n.\n...\n......\n...\n..\n.\n....\n..\n.......\n......\n........\n.........\n.\n......\n......\n......\n..\n.........\n....\n..\n.\n..\n.........\n.....\n..\n..\n..\n..\n.\n.\n....\n.\n......\n..\n........\n.....\n..\n........\n.\n....\n.........\n...\n.............\n..........\n.\n..\n......\n...........\n.\n..\n...\n.....\n...........\n........\n..\n.\n..\n...\n.\n..\n..\n.....\n.....\n.......\n.\n...\n..\n.....\n.........\n.\n.......\n...\n..\n.....\n.......\n...\n..........\n...\n....\n...\n...\n.......\n...\n.\n....\n..\n........\n....\n......\n.........\n..\n.......\n..\n..........\n..\n.\n.\n.\n......\n......\n...\n...\n.\n.......\n.......\n..\n..\n..\n.\n...\n..\n......\n...\n.....\n....\n..\n.\n.....\n...\n..\n.\n......\n.......\n....\n...\n..\n........\n..\n.\n.\n....\n.....\n.\n.\n.\n.\n.......\n.......\n.......\n..\n.\n.\n.\n.....\n...\n....\n.............\n..\n.\n......\n....\n...\n..\n....\n....\n.\n.....\n......\n....\n.....\n.....\n.\n.\n.\n.\n..\n.\n.........\n....\n..\n....\n.......\n....\n.\n.........\n..........\n....\n..\n.......\n...\n...\n.\n...\n....\n..\n.\n.\n.\n......\n..........\n.....\n.\n....\n....\n....\n.\n......\n.\n..\n.........\n......\n..\n..\n.....\n.....\n........\n.\n...............\n......\n..\n.....\n..\n..\n.\n.\n......\n......\n............\n....\n.........\n.\n..\n...........\n..\n..\n....\n.....\n.\n...\n.\n..\n.\n....\n.....\n..\n.\n......\n.\n.......\n......\n.\n.......\n.\n...\n.\n......\n..\n..\n.......\n..\n...\n...\n.\n...\n..\n.\n..............\n.\n.\n.\n..\n.\n....\n.\n.......\n....\n...\n....\n....\n....\n..\n...\n.\n..\n..\n.\n.\n........\n...\n..\n.\n.\n.\n..\n...\n..\n..\n..\n.\n..\n..\n.\n..\n.........\n...........\n.......\n........\n..\n.\n.....\n...\n....\n.....\n.\n...\n........\n.........\n.......\n.\n......\n..\n.\n.\n.\n......\n...\n.\n.......\n.....\n...\n.\n...\n.\n..\n..............\n..\n.......\n...\n...\n.....\n.....\n......\n.......\n..\n.\n............\n......\n............\n..\n.\n..\n.....\n.\n.\n....\n.....\n............\n..\n..\n....\n...\n...\n..\n.\n.......\n......\n....\n......\n......\n....\n.\n.\n...\n.\n....\n.\n.....\n...\n...\n.........\n.\n.\n.\n.\n.....\n..\n.\n.......\n......\n.\n..\n.....\n.\n........\n..\n...\n..\n...\n....\n.....\n..\n.\n......\n....\n..\n...\n.\n................\n......\n.\n...\n.\n..\n......\n....\n.\n..\n......\n.\n..\n..\n....\n....\n.....\n.\n...........\n..\n.\n.....\n.\n........\n..\n.\n...\n...\n..\n.\n..\n..\n..\n...\n.\n......\n..\n........\n........\n.\n...\n.....\n...\n..\n..\n.....\n....\n..\n..\n....\n...\n...\n..\n..\n..\n......\n.\n...\n...\n..\n..\n...\n..\n....\n...\n.\n.....\n.....\n....\n.\n..\n..\n...\n..\n.....\n.\n...\n......\n..\n.\n..\n...\n..\n........\n.\n.\n.\n...\n..\n..\n.\n.......\n.\n.\n.\n....\n..\n..\n.\n....\n..\n.\n...\n....\n.......\n..\n.\n..\n...\n.....\n..\n...\n.\n....\n....\n..\n......\n.\n.\n..\n.\n.\n...\n...\n.\n......\n......\n..\n...\n....\n.\n..\n.....\n...\n...\n..\n.\n.\n.\n.\n.\n.\n..\n..\n...\n.\n..........\n.........\n..\n.\n...\n.....\n...\n.\n.\n.\n.\n.\n.\n..\n....\n...\n....\n..\n...\n.....\n..\n.\n....\n..............\n....\n.....\n.....\n.....\n....\n...\n.\n.\n..\n.....\n..\n.\n....\n..\n.\n...\n.\n...\n.\n...\n....\n.....\n.....\n......\n..\n.\n..\n...\n..\n..\n..\n.\n..\n..\n......\n.........\n....\n........\n.\n......\n......\n.\n..\n...\n.......\n..\n.\n..\n....\n..\n...\n.\n...\n..\n..\n..........\n.\n..\n..\n...\n.\n....\n..\n....\n....\n......\n...\n..\n......\n....\n...\n...\n..\n..\n.\n.\n..\n.....\n.....\n..\n..\n.\n.\n.\n..\n.\n...\n..\n..\n..\n.....\n.\n..\n..\n.\n..........\n....\n.....\n.\n....\n..\n..\n.\n......\n.\n..\n..\n........\n.\n...\n.\n....\n.......\n.\n.\n.\n..........\n..\n.....\n..\n.....\n..\n..\n..\n.\n.\n.....\n..\n.\n.\n.\n...\n.\n....\n....\n..\n...\n..\n......\n.......\n.\n...\n.\n.\n.\n..\n...\n.\n....\n..\n..\n...\n.\n..\n..\n.\n..\n.\n...\n..\n.......\n.....\n.\n..\n..\n.\n..\n..\n.....\n....\n....\n.\n..\n.\n...........\n...\n.\n...\n..\n..\n...\n..\n..\n..\n.\n..\n...\n...\n......\n.....\n..\n.\n........\n..............\n..\n.\n.\n.\n....\n..\n...\n....\n...\n..\n.\n..\n...\n..\n.\n...\n..\n..\n.\n.....\n.\n..\n.\n.\n.\n.\n...\n.\n...\n.\n........\n.....\n..\n.\n...\n..\n..\n.\n....\n..\n...\n...\n...\n.\n...\n.\n.\n....\n.\n..\n.....\n.\n.....\n.\n.\n.\n..\n..\n..\n..\n.\n.\n..\n.\n.\n.\n....\n..\n..\n..\n.\n....\n..\n....\n..\n...\n.\n..\n.\n.\n......\n.........\n.....\n....\n.\n...\n...\n..\n.\n....\n...\n.\n.\n...\n.\n.\n..\n....\n..\n.......\n.\n.\n.\n.\n.\n.\n..\n..\n....\n...\n.\n.\n.\n..\n.\n...\n.\n.\n....\n...\n.\n...\n....\n.\n....\n.\n..\n..\n....\n..\n..\n.\n..\n.\n..\n.\n......\n..\n......\n..\n..\n..\n..\n......\n..\n.\n....\n.\n.\n....\n..\n....\n.\n...\n..\n.\n.\n..\n..\n.....\n.\n...\n.\n..\n.\n.....\n..\n..\n..\n..\n...\n.......\n.\n.\n.\n.\n...\n.\n......\n...\n..\n..\n.....\n.\n..\n...\n.\n...\n.\n.\n.......\n.....\n...\n....\n.\n.\n..\n..\n.\n...\n.\n.\n.\n.\n..\n...\n...\n......\n..\n...\n.\n.\n..\n.\n..\n....\n.\n..\n..\n...\n.\n.\n...\n..\n.\n...\n...\n.\n.\n.\n.\n...\n.\n...\n.\n..\n.\n..\n......\n.\n.....\n..\n..\n...\n...\n..\n.\n...\n.\n.\n.....\n...\n.\n..\n..\n.\n......\n...\n.\n........\n..\n..\n......\n..\n.\n.\n.\n...\n.\n.\n.\n...\n.\n..\n..\n.\n....\n..\n.\n.\n..\n...\n..\n.\n..\n....\n..\n..\n.\n.\n.\n......\n.......\n...\n..\n.\n.\n.\n.\n.....\n.\n.\n.\n...\n.\n.\n...\n...\n.....\n..\n..\n..\n.\n..\n.\n..\n......\n....\n....\n....\n.\n.\n...\n...\n...\n.........\n..\n.\n....\n....\n.\n.....\n...\n....\n..\n...\n.\n.\n.\n.\n..\n......\n.\n.\n.....\n.\n..\n..\n..\n..\n...\n..\n.\n.\n.\n.\n....\n...\n.\n.\n..\n.\n.\n..\n.\n....\n...\n.\n.\n.\n.\n.\n...\n..\n.\n..\n.\n..\n..\n.\n.\n..\n.\n..\n...\n....\n.\n.\n.\n....\n.\n.....\n.\n...\n.\n...\n...\n..\n.\n.\n..\n..\n.\n.\n...\n.\n.\n...\n.\n..\n...\n.\n.\n.\n.\n.\n.\n...\n..\n...\n.\n..\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.....\n.\n.\n.\n..\n.\n..\n.\n.\n....\n..\n.\n.\n.\n..\n....\n..\n....\n.\n.\n.\n.\n.\n.\n...\n....\n.\n.\n..\n.\n.\n.\n...\n....\n...\n...\n.\n...\n.\n..\n...\n...\n....\n.\n...\n...\n.\n...\n.\n.\n...\n.\n.\n.\n..\n.\n..\n..\n.\n.\n...\n..\n.\n.\n.\n.\n..\n..\n..\n.\n.\n.\n.\n.\n..\n.\n..\n...\n..\n..\n..\n.\n..\n.\n..\n.\n...\n.\n.\n..\n..\n.\n..\n.\n..\n.\n..\n....\n.\n.\n..\n.\n..\n..\n.\n.\n.\n.....\n.\n...\n....\n.\n.\n...\n.\n..\n....\n.\n..\n.\n.\n.\n.\n..\n.\n..\n..\n...\n.\n...\n..\n..\n..\n..\n.\n.\n...\n.\n.\n...\n..\n.\n.\n.....\n.\n..\n.\n..\n.\n...\n.\n.\n..\n..\n.\n.\n.\n.\n..\n.\n...\n.\n..\n.\n.\n..\n...\n..\n.\n.\n..\n..\n...\n..\n.\n.\n.\n..\n...\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n....\n..\n.\n..\n.\n.\n.\n.\n...\n..\n.\n...\n...\n.\n..\n..\n..\n.\n..\n.\n.\n.\n.\n.\n..\n...\n....\n.\n.\n...\n..\n.\n..\n.\n..\n.\n.\n.\n.\n....\n..\n.\n..\n...\n...\n..\n....\n.\n..\n.....\n.\n..\n.....\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.....\n.\n..\n.\n.\n....\n.\n.\n..\n.\n.\n.\n.\n...\n.\n.\n.\n..\n..\n.\n.\n...\n..\n..\n...\n.\n..\n.\n.\n..\n.\n.\n..\n.\n...\n.\n.\n..\n..\n.\n..\n.\n..\n.\n.\n..\n.\n...\n.\n..\n.\n.....\n.\n..\n..\n.\n..\n.\n...\n..\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n..\n..\n.\n.\n..\n.\n.\n.\n...\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n...\n.\n.\n........\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n..\n.\n.\n.\n...\n...\n.\n.\n.\n..\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n...\n.\n..\n.\n.\n.\n.\n....\n.\n.\n.\n.\n.\n..\n.\n.\n..\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n...\n.\n.\n...\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n..\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nORDER\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n... .\n.\n. ..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n...\n.\n.\n.\n....\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n..\n.\n..\n... .\n...\n..\n...\n.\n.\n.\n.\n..\n.\n.\n.\n....\n.\n.\n.\n.\n.\n.. .\n.\n.\n.\n.\n.\n.\n...\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n....\n.\n.\n..\n.\n.\n.\n.\n. .\n.\n..\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n....\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n....\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n. .\n.\n..\n. .\nphilosophy/history of science\n.\n.\n.\n.\n.\n.\n....\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n...\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n. ...\n.\n.\n..\n.\n...\n..\n.\n.\n..\n..\n.... .\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.....\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n....\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n....\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nPROBLEM\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\nDISCOVERY\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\nORIGIN\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\ngeology\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nSTONE\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\nERA\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\nPROPERTY\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n. .\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.......\n.\n.\n.\n.\n.\n.\n..\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n. ..\n.\n.\n.\n.\n..\n.\n.\n....\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\nbanking/insurance\n.\n.\n.\n.\n.\n.\n.\n.\n..\n....\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n..\n.\n..\n.\n.\n.\n.\n....\n.\n.\n.\n.\n..\n.\n..\n..\n.\n...\n.\n.\n..\n..\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n...\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n..\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n..\n.\n...\n.\n.\n.\n.\n.\n..\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n...\n..\n..\n..\n.\n.\n...\n.\n..\n.\n..\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.......\n.\n.\n.\n.\n.\n.\n..\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n..\n..\n.\n...\n.\n.\n..\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n...\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n......\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n...\n.\n.\n.\n.\n.\n.\n....\n.\n.\n.\n....\n.\n.\n.\n.\n..\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n..\n..\n.\n..\n..\n..\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n.\n..\n.\n.\n.\n.....\n.\n.\n.\n..\n.\n.\n.\n.\n.\n....\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.....\n.\n.\n..\n..\n...\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n...\n.\n..\n.\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n...\n.....\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n....\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n..\n..\n..\n.\n....\n.\n.\n.....\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n....\n..\n..\n..\n...\n.\n.\n.\n.\n.\n.\n.\n...\n.\n.\n..\n..\n...\n..\n..\n.\n.\n.\n.\n.\n.\n..\n.\n...\n.\n.\n.\n.\n.\n.\n.\n.\n..\n...\n.\n.\n.\n..\n.\n.\n.\n..\n..\n...\n..\n....\n.\n.\n...\n.\n.\n.\n.\n.\n...\n..\n.\n.\n.\n..\n.\n.\n.\n..\n...\n.\n......\n...\n.\n.\n.\n.....\n.\n..\n.\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n..\n.\n....\n..\n..\n...\n.....\n.\n.\n.\n...\n..\n.\n.\n..\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n..\n......\n.\n.\n..\n....\n..\n..\n..\n..\n.\n.\n.\n..\n..\n.\n...\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n...\n...\n.\n.\n....\n.\n.\n.\n...\n.\n.\n.\n..\n.\n.\n...\n.\n.\n.\n..\n.\n.....\n.\n.\n..\n.\n.\n.\n..\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n......\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n..\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n..\n.\n.\n.\n.\n...\n.\n.\n.\n..\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n.\n\nEXPOSED\nSUBSTANTIALLY\n\nPRICE\nMONEY\nINSURANCE\n\nGUIDED\n\nREPRESENTED\n\nSELECTIONVISION\n\nAGENTS\n\nDESTROYED\n\nDENSE\nDROPPED\n\nCOLLECTS\n\nBILLIONS\n\n.\n.\n.\n.\n.\n.\n.\n\nCOMPREHENSIVE\n\nBLENDS\n\n.\n.\n\n.\n.\n\nACCUMULATION\n\nRETAINED\n\nDEPOSITS\nRISE\n\nOWES\n\nDUE\n\nESTIMATE\n\nABSENT\n\nFIRE\n\nUPPER\n\nAVERAGE\n\nINTEREST\n\nESSENTIALS\n\nSHAPED\n\nREQUIRES\n\nTHOUSANDS\n\nCALLED\n\nFOUND\n\nENABLE\n\nHUMAN\n\nSCIENTIFIC\n\nHYPOTHESIS\n\nPERMEABLE\n\nDRILLED\n\nFOLD\n\nDOME\n\n.\n.\n.\n.\n\nTAX\n\nFigure 3: Parametric embedding for word meanings and topics based on posterior distri-\nbutions from an LDA model. Each dot represents the coordinates rn of one word. Large\nphrases indicate the positions of topic means (cid:30)k (with topics labeled intuitively). Examples\nof words that belong to one or more topics are also shown.\n\nover word types that can occur in a document. This model can be inverted to give the prob-\nability that topic ck was responsible for generating word xn; these probabilities p(ckjxn)\nprovide the input needed to construct a space of word and topic meanings in PE.\n\nMore speci\ufb01cally, we \ufb01t a 50-topic LDA model to the TASA corpus. Then, for each word\ntype, we computed its posterior distribution restricted to a subset of 5 topics, and input these\nconditional probabilities to PE (with N = 26; 243, K = 5). Fig. 3 shows the resulting\nembedding. As with the embeddings in Figs. 1 and 2, the topics are arranged roughly in a\nstar shape, with a tight cluster of points at each corner of the star corresponding to words\nthat place almost all of their probability mass on that topic. Semantically, the words in these\nextreme clusters often (though not always) have a fairly specialized meaning particular to\nthe nearest topic. Moving towards the center of the plot, words take on increasingly general\nmeanings.\n\nThis embedding shows other structures not visible in previous \ufb01gures: in particular, dense\ncurves of points connecting every pair of clusters. This pattern re\ufb02ects the characteristic\nprobabilistic structure of topic models of semantics: in addition to the clusters of words\nthat associate with just one topic, there are many words that associate with just two topics,\nor just three, and so on. The dense curves in Fig. 3 show that for any pair of topics in\nthis corpus, there exists a substantial subset of words that associate with just those topics.\n\n\fFor words with probability sharply concentrated on two topics, points along these curves\nminimize the sum of the KL and regularization terms. This kind of distribution is intrinsi-\ncally high-dimensional and cannot be captured with complete \ufb01delity in any 2-dimensional\nembedding.\n\nAs shown by the examples labeled in Fig. 3, points along the curves connecting two appar-\nently unrelated topics often have multiple meanings or senses that join them to each topic:\n\u2018deposit\u2019 has both a geological and a \ufb01nancial sense, \u2018phase\u2019 has both an everyday and a\nchemical sense, and so on.\n\n6 Conclusions\n\nWe have proposed a probabilistic embedding method, PE, that embeds objects and classes\nsimultaneously. PE takes as input a probability distribution for objects over classes, or more\ngenerally of one set of points over another set, and attempts to \ufb01t that distribution with a\nsimple class-conditional parametric mixture in the embedding space. Computationally, PE\nis inexpensive relative to methods based on similarities or distances between all pairs of\nobjects, and converges quickly on many thousands of data points.\n\nThe visualization results of PE shed light on features of both the data set and the classi\ufb01-\ncation model used to generate the input conditional probabilities, as shown in applications\nto classi\ufb01ed web pages, partially classi\ufb01ed digits, and the latent topics discovered by an\nunsupervised method, LDA. PE may also prove useful for similarity-preserving dimension\nreduction, where the high-dimensional model is not of primary interest, or more generally,\nin analysis of large conditional probability tables that arise in a range of applied domains.\n\nAs an example of an application we have not yet explored, purchases, web-sur\ufb01ng histo-\nries, and other preference data naturally form distributions over items or categories of items.\nConversely, items de\ufb01ne distributions over people or categories thereof. Instances of such\ndyadic data abound\u2013restaurants and patrons, readers and books, authors and publications,\nspecies and foods...\u2013with patterns that might be visualized. PE provides a tractable, princi-\npled, and effective visualization method for large volumes of such data, for which pairwise\nmethods are not appropriate.\n\nAcknowledgments\n\nThis work was supported by a grant from the NTT Communication Sciences Laboratories.\n\nReferences\n[1] D. Blei, A. Ng and M. Jordan. Latent dirichlet allocation. NIPS 15, 2002.\n[2] V. de Silva, J. B. Tenenbaum. Global versus local methods in nonlinear dimensionality\n\nreduction. NIPS 15, pp. 705-712, 2002.\n\n[3] R. Fisher. The use of multiple measurements in taxonomic problem. Annuals of Eu-\n\ngenics 7, pp.179\u2013188, 1950.\n\n[4] G. Hinton and S. Roweis. Stochastic neighbor embedding. NIPS 15, 2002.\n[5] I.T. Joliffe. Principal Component Analysis. Springer, 1980.\n[6] J. Tenenbaum, V. de Silva and J. Langford. A global geometric framework for non-\n\nlinear dimensionality reduction. Science 290 pp. 2319\u20132323, 2000.\n\n[7] W. Torgerson. Theory and Methods of Scaling. New York, Wiley, 1958.\n\n\f", "award": [], "sourceid": 2556, "authors": [{"given_name": "Tomoharu", "family_name": "Iwata", "institution": null}, {"given_name": "Kazumi", "family_name": "Saito", "institution": null}, {"given_name": "Naonori", "family_name": "Ueda", "institution": null}, {"given_name": "Sean", "family_name": "Stromsten", "institution": null}, {"given_name": "Thomas", "family_name": "Griffiths", "institution": null}, {"given_name": "Joshua", "family_name": "Tenenbaum", "institution": null}]}