{"title": "Automated scalable segmentation of neurons from multispectral images", "book": "Advances in Neural Information Processing Systems", "page_first": 1912, "page_last": 1920, "abstract": "Reconstruction of neuroanatomy is a fundamental problem in neuroscience. Stochastic expression of colors in individual cells is a promising tool, although its use in the nervous system has been limited due to various sources of variability in expression. Moreover, the intermingled anatomy of neuronal trees is challenging for existing segmentation algorithms. Here, we propose a method to automate the segmentation of neurons in such (potentially pseudo-colored) images. The method uses spatio-color relations between the voxels, generates supervoxels to reduce the problem size by four orders of magnitude before the final segmentation, and is parallelizable over the supervoxels. To quantify performance and gain insight, we generate simulated images, where the noise level and characteristics, the density of expression, and the number of fluorophore types are variable. We also present segmentations of real Brainbow images of the mouse hippocampus, which reveal many of the dendritic segments.", "full_text": "Automated scalable segmentation of neurons from\n\nmultispectral images\n\nUygar S\u00fcmb\u00fcl\n\nGrossman Center for the Statistics of Mind\nand Dept. of Statistics, Columbia University\n\nDouglas Roossien Jr.\n\nUniversity of Michigan Medical School\n\nFei Chen\n\nMIT Media Lab and McGovern Institute\n\nNicholas Barry\n\nMIT Media Lab and McGovern Institute\n\nEdward S. Boyden\n\nMIT Media Lab and McGovern Institute\n\nDawen Cai\n\nUniversity of Michigan Medical School\n\nJohn P. Cunningham\n\nGrossman Center for the Statistics of Mind\nand Dept. of Statistics, Columbia University\n\nLiam Paninski\n\nGrossman Center for the Statistics of Mind\nand Dept. of Statistics, Columbia University\n\nAbstract\n\nReconstruction of neuroanatomy is a fundamental problem in neuroscience.\nStochastic expression of colors in individual cells is a promising tool, although its\nuse in the nervous system has been limited due to various sources of variability in\nexpression. Moreover, the intermingled anatomy of neuronal trees is challenging\nfor existing segmentation algorithms. Here, we propose a method to automate the\nsegmentation of neurons in such (potentially pseudo-colored) images. The method\nuses spatio-color relations between the voxels, generates supervoxels to reduce\nthe problem size by four orders of magnitude before the \ufb01nal segmentation, and is\nparallelizable over the supervoxels. To quantify performance and gain insight, we\ngenerate simulated images, where the noise level and characteristics, the density\nof expression, and the number of \ufb02uorophore types are variable. We also present\nsegmentations of real Brainbow images of the mouse hippocampus, which reveal\nmany of the dendritic segments.\n\nIntroduction\n\n1\nStudying the anatomy of individual neurons and the circuits they form is a classical approach\nto understanding how nervous systems function since Ram\u00f3n y Cajal\u2019s founding work. Despite\na century of research, the problem remains open due to a lack of technological tools: mapping\nneuronal structures requires a large \ufb01eld of view, a high resolution, a robust labeling technique, and\ncomputational methods to sort the data. Stochastic labeling methods have been developed to endow\nindividual neurons with color tags [1, 2]. This approach to neural circuit mapping can utilize the\nlight microscope, provides a high-throughput and the potential to monitor the circuits over time, and\ncomplements the dense, small scale connectomic studies using electron microscopy [3] with its large\n\ufb01eld-of-view. However, its use has been limited due to its reliance on manual segmentation.\nThe initial stochastic, spectral labeling (Brainbow) method had a number of limitations for neuro-\nscience applications including incomplete \ufb01lling of neuronal arbors, disproportionate expression\nof the nonrecombined \ufb02uorescent proteins in the transgene, suboptimal \ufb02uorescence intensity, and\ncolor shift during imaging. Many of these limitations have since improved [4] and developments\n\n30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain.\n\n\fin various aspects of light microscopy provide further opportunities [5, 6, 7, 8]. Moreover, recent\napproaches promise a dramatic increase in the number of (pseudo) color sources [9, 10, 11]. Taken\ntogether, these advances have made light microscopy a much more powerful tool for neuroanatomy\nand connectomics. However, existing automated segmentation methods are inadequate due to the\nspatio-color nature of the problem, the size of the images, and the complicated anatomy of neuronal\narbors. Scalable methods that take into account the high-dimensional nature of the problem are\nneeded.\nHere, we propose a series of operations to segment 3-D images of stochastically tagged nervous\ntissues. Fundamentally, the computational problem arises due to insuf\ufb01cient color consistency within\nindividual cells, and the voxels occupied by more than one neuron. We denoise the image stack\nthrough collaborative \ufb01ltering [12], and obtain a supervoxel representation that reduces the problem\nsize by four orders of magnitude. We consider the segmentation of neurons as a graph segmentation\nproblem [13], where the nodes are the supervoxels. Spatial discontinuities and color inhomogeneities\nwithin segmented neurons are penalized using this graph representation. While we concentrate on\nneuron segmentation in this paper, our method should be equally applicable to the segmentation of\nother cell classes such as glia.\nTo study various aspects of stochastic multispectral labeling, we present a basic simulation algorithm\nthat starts from actual single neuron reconstructions. We apply our method on such simulated images\nof retinal ganglion cells, and on two different real Brainbow images of hippocampal neurons, where\none dataset is obtained by expansion microscopy [5].\n2 Methods\nSuccessful segmentations of color-coded neural images should consider both the connected nature\nof neuronal anatomy and the color consistency of the Brainbow construct. However, the size and\nthe noise level of the problem prohibit a voxel-level approach (Fig. 1). Methods that are popular in\nhyperspectral imaging applications, such as nonnegative matrix factorization [14], are not immediately\nsuitable either because the number of color channels are too few and it is not easy to model neuronal\nanatomy within these frameworks. Therefore, we develop (i) a supervoxelization strategy, (ii)\nexplicitly de\ufb01ne graph representations on the set of supervoxels, and (iii) design the edge weights to\ncapture the spatio-color relations (Fig. 2a).\n2.1 Denoising the image stack\nVoxel colors within a neurite can drift along the neurite, exhibit high frequency variations, and differ\nbetween the membrane and the cytoplasm when the expressed \ufb02uorescent protein is membrane-\nbinding (Fig. 1). Collaborative \ufb01ltering generates an extra dimension consisting of similar patches\nwithin the stack, and applies \ufb01ltering in this extra dimension rather than the physical dimensions.\nWe use the BM4D denoiser [12] on individual channels of the datasets, assuming that the noise is\nGaussian. Figure 2 demonstrates that the boundaries are preserved in the denoised image.\n2.2 Dimensionality reduction\nWe make two basic observations to reduce the size of the dataset: (i) Voxels expressing \ufb02uorescent\nproteins form the foreground, and the dark voxels form the much larger background in typical\nBrainbow settings. (ii) The basic promise of Brainbow suggests that nearby voxels within a neurite\nhave very similar colors. Hence, after denoising, there must be many topologically connected voxel\nsets that also have consistent colors.\nThe watershed transform [15] considers its input as a topographic map and identi\ufb01es regions associated\nwith local minima (\u201ccatchment basins\u201d in a \ufb02ooding interpretation of the topographic map). It can\nbe considered as a minimum spanning forest algorithm, and obtained in linear time with respect\nto the input size [16, 17]. For an image volume V = V (x, y, z, c), we propose to calculate the\ntopographical map T (disaf\ufb01nity map) as\n\nT (x, y, z) = max\n\nt\u2208{x,y,z} max\n\nc\n\n|Gt(x, y, z, c)|,\n\n(1)\n\nwhere x, y, z denote the spatial coordinates, c denotes the color coordinate, and Gx, Gy, Gz denote\nthe spatial gradients of V (nearest neighbor differencing). That is, any edge with signi\ufb01cant deviation\nin any color channel will correspond to a \u201cmountain\u201d in the topographic map. A \ufb02ooding parameter,\nf, assigns the local minima of T to catchment basins, which partition V together with the boundary\nvoxels. We assign the boundaries to neighboring basins based on color proximity. The background is\n\n2\n\n\fFigure 1: Multiple noise sources affect the color consistency in Brainbow images. a, An 85\u00d7121\nBrainbow image patch from a single slice (physical size: 8.5\u00b5 \u00d7 12.1\u00b5). Expression level differs\nsigni\ufb01cantly between the membrane and the cytoplasm along a neurite (arrows). b, A maximum\nintensity projection view of the 3-d image stack. Color shifts along a single neurite, which travels\nto the top edge and into the page (arrows). c, A 300 \u00d7 300 image patch from a single slice of a\ndifferent Brainbow image (physical size: 30\u00b5 \u00d7 30\u00b5). d, The intensity variations of the different\ncolor channels along the horizontal line in c. e, Same as d for the vertical line in c. f, The image\npatch in c after denoising. g\u2013h, Same as d and e after denoising. For the plots, the range of individual\ncolor channels is [0, 1].\n\nthe largest and darkest basin. We call the remaining objects supervoxels [18, 19]. Let F denote the\nbinary image identifying all of the foreground voxels.\nObjects without interior voxels (e.g., single-voxel thick dendritic segments) may not be detected by\nEq. 1 (Supp. Fig. 1). We recover such \u201cbridges\u201d using a topology-preserving warping (in this case,\nonly shrinking is used.) of the thresholded image stack into F [20, 21]:\n\nB = W(I\u03b8, F ),\n\n(2)\nwhere I\u03b8 is binary and obtained by thresholding the intensity image at \u03b8. W returns a binary image\nB such that B has the same topology as I\u03b8 and agrees with F as much as possible. Each connected\ncomponent of B \u2227 \u00afF (foreground of B and background of F ) is added to a neighboring supervoxel\nbased on color proximity, and discarded if no spatial neighbors exist (Supp. Text).\nWe ensure the color homogeneity within supervoxels by dividing non-homogeneous supervoxels (e.g.,\nlarge color variation across voxels) into connected subcomponents based on color until the desired\nhomogeneity is achieved (Supp. Text). We summarize each supervoxel\u2019s color by its mean color.\nWe apply local heuristics and spatio-color constraints iteratively to further reduce the data size and\ndemix overlapping neurons in voxel space (Fig. 2f,g and Supp. Text). Supp. Text provides details on\nthe parallelization and complexity of these steps and the method in general.\n\n3\n\nBACDEFGH05101500.10.20.30.40.50.60.70.80.91x\u2212position (\u00b5m)normalized intensity0246810121400.10.20.30.40.50.60.70.8y\u2212position (\u00b5m)normalized intensity0246810121400.10.20.30.40.50.60.7y\u2212position (\u00b5m)normalized intensity05101500.10.20.30.40.50.60.70.80.91x\u2212position (\u00b5m)normalized intensity\fof\n\nFigure 2: Best\nviewed digitally.\na, A schematic\nof the processing\nsteps b, Max.\nintensity\npro-\njection of a raw\nBrainbow image\nc, Max. intensity\nprojection of the\ndenoised\nimage\nd, A zoomed-in\nversion\nthe\npatch indicated by\nthe dashed square\nin b.\ne, The\ncorresponding\ndenoised image. f,\nOne-third of the\nsupervoxels in the\ntop-left quadrant\n(randomly\ncho-\nsen). g, Same as f\nafter the merging\nh1-h4,\nstep.\nSame as b,c,f,g\nfor\nsimulated\ndata. Scale bars,\n20\u00b5m.\n\n2.3 Clustering the supervoxel set\nWe consider the supervoxels as the nodes of a graph and express their spatio-color similarities\nthrough the existence (and the strength) of the edges connecting them, summarized by a highly\nsparse adjacency matrix. Removing edges between supervoxels that aren\u2019t spatio-color neighbors\navoids spurious links. However, this procedure also removes many genuine links due to high color\nvariability (Fig. 1). Moreover, it cannot identify disconnected segments of the same neuron (e.g., due\nto limited \ufb01eld-of-view). Instead, we adjust the spatio-color neighborhoods based on the \u201creliability\u201d\nof the colors of the supervoxels. Let S denote the set of supervoxels in the dataset. We de\ufb01ne\nthe sets of reliable and unreliable supervoxels as Sr = {s \u2208 S : n(s) > ts, h(s) < td} and\nSu = S \\ Sr, respectively, where n(s) denotes the number of voxels in s, h(s) is a measure of the\ncolor heterogeneity (e.g., the maximum difference between intensities across all color channels), ts\nand td are the corresponding thresholds.\nWe describe a graph G = (V, E), where V denotes the vertex set (supervoxels) and E = Es\u222aEc\u222aE\u00afs\ndenotes the edges between them:\n\nEs = {(ij) : \u03b4ij < \u0001s, i (cid:54)= j}\nEc = {(ij) : si, sj \u2208 Sr, dij < \u0001c, i (cid:54)= j}\nE\u00afs = {(ij), (ji) : si \u2208 Su, (ij) /\u2208 Es, Oi(j) < kmin \u2212 Ki, i (cid:54)= j},\n\n(3)\nwhere \u03b4ij, dij are the spatial and color distances between si and sj, respectively. \u0001s and \u0001c are\nthe corresponding maximum distances. An unreliable supervoxel with too few spatial neighbors is\nallowed to have up to kmin edges via proximity in color space. Here, Oi(j) is the order of supervoxel\nsj in terms of the color distance from supervoxel si, and Ki is the number of \u0001s-spatial neighbors of\nsi. (Note the symmetric formulation in E\u00afs.) Then, we construct the adjacency matrix as\n\n(4)\n\nA(i, j) =\n\n(ij) \u2208 E\notherwise\n\n(cid:26)\n\ne\u2212\u03b1d2\nij ,\n0,\n\n4\n\n\fwhere \u03b1 controls the decay in af\ufb01nity with respect to distance in color. We use k-d tree structures\nto ef\ufb01ciently retrieve the color neighborhoods [22]. Here, the distance between two supervoxels is\nminv\u2208V,u\u2208U D(v, u), where V and U are the voxel sets of the two supervoxels and D(v, u) is the\nEuclidean distance between voxels v and u.\nA classical way of partitioning graph nodes that are nonlinearly separable is by minimizing a function\n(e.g., the sum or the maximum) of the edge weights that are severed during the partitioning [23].\nHere, we use the normalized cuts algorithm [24, 13] with two simple modi\ufb01cations: the k-means step\nis weighted by the sizes of the supervoxels and initialized by a few iterations of k-means clustering\nof the supervoxel colors only (Supp. Text). The resulting clusters partition the image stack (together\nwith the background), and represent a segmentation of the individual neurons within the image stack.\nAn estimate of the number of neurons can be obtained from a Dirichlet process mixture model [25].\nWhile this estimate is often rough [26], the segmentation accuracy appears resilient to imperfect\nestimates (Fig. 4c).\n2.4 Simulating Brainbow tissues\nWe create basic simulated Brainbow image stacks from volumetric reconstructions of single neurons\n(Algorithm 1). For simplicity, we model the neuron color shifts by a Brownian noise component on\nthe tree, and the background intensity by a white Gaussian noise component (Supp. Text).\nWe quantify the segmentation quality of the voxels using the adjusted Rand index (ARI), whose\nmaximum value is 1 (perfect agreement), and expected value is 0 for random clusters [27]. (Supp.\nText)\n\nbackground noise variability \u03c31, neural color variability \u03c32, r, saturation level M\n\nShift and rotate neuron ni to minimize overlap with existing neurons in the stack\nGenerate a uniformly random color vector vi of length C\nIdentify the connected components of cij of ni within the stack\nfor cij \u2208 {cij}j do\n\nAlgorithm 1 Brainbow image stack simulation\nRequire: number of color channels C, set of neural shapes S = {ni}i, stack (empty, 3d space + color),\n1: for ni \u2208 S do\n2:\n3:\n4:\n5:\n6:\n7:\n8:\n9:\n10: end for\n11: Add white noise to each voxel generated by N (0, \u03c32\n12: if brightness exceeds M then\n13:\n14: end if\n15: return stack\n\nSaturate at M\n\nPre-assign vi to r% of the voxels of cij\nC-dimensional random walk on cij with steps N (0, \u03c32\n\nend for\nAdd neuron ni to the stack (with additive colors for shared voxels)\n\n1I) (Supp. Text)\n\n2I)\n\n3 Datasets\nTo simulate Brainbow image stacks, we used volumetric single neuron reconstructions of mouse\nretinal ganglion cells in Algorithm 1. The dataset is obtained from previously published studies [28,\n29]. Brie\ufb02y, the voxel size of the images is 0.4\u00b5 \u00d7 0.4\u00b5 \u00d7 0.5\u00b5, and the \ufb01eld of view of individual\nstacks is 320\u00b5 \u00d7 320\u00b5 \u00d7 70\u00b5 or larger. We evaluate the effects of different conditions on a central\nportion of the simulated image stack.\nBoth real datasets are images of the mouse hippocampal tissue. The \ufb01rst dataset has 1020\u00d71020\u00d7225\nvoxels (voxel size: 0.1\u00d7 0.1\u00d7 0.3\u00b53), and the tissue was imaged at 4 different frequencies (channels).\nThe second dataset has 1080 \u00d7 1280 \u00d7 134 voxels with an effective voxel size of 70 \u00d7 70 \u00d7 40nm,\nwhere the tissue was 4\u00d7 linearly expanded [5], and imaged at 3 different channels. The Brainbow\nconstructs were delivered virally, and approximately 5% of the neurons express a \ufb02uorescence gene.\n4 Results\nParameters used in the experiments are reported in Supp. Text.\nFig. 1b, d, and e depict the variability of color within individual neurites in a single slice and through\nthe imaging plane. Together, they demonstrate that the voxel colors of even a small segment of a\n\n5\n\n\fFigure 3: Segmentation\nof a simulated Brainbow\nimage stack. Adjusted\nRand index of the fore-\nground is 0.80. Pseudo-\ncolor representation of 4-\nchannel data. Top: max-\nimum intensity projection\nof the ground truth. Only\nthe supervoxels that are oc-\ncupied by a single neuron\nare shown. Bottom: max-\nimum intensity projection\nof the reconstruction. The\ntop-left corners show the\nwhole image stack. All\nother panels show the max-\nimum intensity projections\nof the supervoxels assigned\nto a single cluster (inferred\nneuron).\n\nFigure 4: Segmentation accuracy of simulated data a, Expression density (ratio of voxels occupied\nby at least one neuron) vs. ARI. b, \u03c31 (Algorithm 1) vs. ARI. c, Channel count vs. ARI for a 9-neuron\nsimulation, where K \u2208 [6, 12]. ARI is calculated for the foreground voxels. See Supp. Fig. 7 for\nARI values for all voxels.\n\nneuron\u2019s arbor can occupy a signi\ufb01cant portion of the dynamic range in color with the state-of-the-\nart Brainbow data. Fig. 1c-e show that collaborative denoising removes much of this noise while\npreserving the edges, which is crucial for segmentation. Fig. 2b-e and h demonstrate a similar effect\non a larger scale with real and simulated Brainbow images.\nFig. 2 shows the raw and denoised versions of the 1020 \u00d7 1020 \u00d7 225 image, and a randomly chosen\nsubset of its supervoxels (one-third). The original set had 6.2 \u00d7 104 supervoxels, and the merging\nroutine decreased this number to 3.9 \u00d7 104. The individual supervoxels grew in size while avoiding\nmergers with supervoxels of different neurons. This set of supervoxels, together with a (sparse)\nspatial connectivity matrix, characterizes the image stack. Similar reductions are obtained for all the\nreal and simulated datasets.\nFig. 3 shows the segmentation of a simulated 200\u00d7200\u00d7100 (physical size: 80\u00b5\u00d780\u00b5\u00d750\u00b5) image\npatch. (Supp. Fig. 2 shows all three projections, and Supp. Fig. 3 shows the density plot through\nthe z-axis.) In this particular example, the number of neurons within the image is 9, \u03c31 = 0.04,\n\u03c32 = 0.1, and the simulated tissue is imaged using 4 independent channels. Supp. Fig. 4 shows a\npatch from a single slice to visualize the amount of noise. The segmentation has an adjusted Rand\nindex of 0.80 when calculated for the detected foreground voxels, and 0.73 when calculated for all\nvoxels. (In some cases, the value based on all voxels is higher.) The ground truth image displays only\nthose supervoxels all of whose voxels belong to a single neuron. The bottom part of Fig. 3 shows\n\n6\n\n3450.50.550.60.650.70.750.80.850.9channel counttrue (9)67810111200.020.040.060.080.10.550.60.650.70.750.80.850.90.95step size (\u03c31) \u2212\u2212 range per channel: [0, 1]3 ch.4 ch.5 ch.0.060.080.10.120.140.160.180.650.70.750.80.850.90.951expression density (ratio of occupied voxels)adjusted Rand index3 ch.4 ch.5 ch.\fFigure 5: Segmentation of a Brainbow stack \u2013 best viewed digitally. Pseudo-color represen-\ntation of 4-channel data. The physical size of the stack is 102\u00b5 \u00d7 102\u00b5 \u00d7 68\u00b5. The top-left\ncorner shows the maximum intensity projection of the whole image stack, all other panels show the\nmaximum intensity projections of the supervoxels assigned to a single cluster (inferred neuron).\n\n7\n\n\fthat many of these supervoxels are correctly clustered to preserve the connectivity of neuronal arbors.\nThere are two important mistakes in clusters 4 (merger) and 9 (spurious cluster). These are caused by\naggressive merging of supervoxels (Supp. Fig. 5), and the segmentation quality improves with the\ninclusion of an extra imaging channel and more conservative merging (Supp. Fig. 6). We plot the\nperformance of our method under different conditions in Fig. 4 (and Supp. Fig. 7). We set the noise\nstandard deviation to \u03c31 in the denoiser, and ignored the contribution of \u03c32. Increasing the number\nof observation channels improves the segmentation performance. The clustering accuracy degrades\ngradually with increasing neuron-color noise (\u03c31) in the reported range (Fig. 4b). The accuracy does\nnot seem to degrade when the cluster count is mildly overestimated, while it decays quickly when the\ncount is underestimated (Fig. 4c).\nFig. 5 displays the segmentation of the 1020 \u00d7 1020 \u00d7 225 image. While some mistakes can be\nspotted by eye, most of the neurites can be identi\ufb01ed and simple tracing tools can be used to obtain\n\ufb01nal skeletons/segmentations [30, 31]. In particular, the identi\ufb01ed clusters exhibit homogeneous\ncolors and dendritic pieces that either form connected components or miss small pieces that do not\npreclude the use of those tracing tools. Some clusters appear empty while a few others seem to\ncomprise segments from more than one neuron, in line with the simulation image (Fig. 2.4).\nSupp. Fig. 8 displays the segmentation of the 4\u00d7 expanded, 1080\u00d7 1280\u00d7 134 image. While the two\nreal datasets have different characteristics and voxel sizes, we used essentially the same parameters\nfor both of them throughout denoising, supervoxelization, merging, and clustering (Supp. Text).\nSimilar to Fig. 5, many of the processes can be identi\ufb01ed easily. On the other hand, Supp. Fig. 8\nappears more fragmented, which can be explained by the smaller number of color channels (Fig. 4).\n5 Discussion\nTagging individual cells with (pseudo)colors stochastically is an important tool in biological sciences.\nThe versatility of genetic tools for tagging synapses or cell types and the large \ufb01eld-of-view of light\nmicroscopy positions multispectral labeling as a complementary approach to electron microscopy\nbased, small-scale, dense reconstructions [3]. However, its use in neuroscience has been limited due\nto various sources of variability in expression. Here, we demonstrate that automated segmentation of\nneurons in such image stacks is possible. Our approach considers both accuracy and scalability as\ndesign goals.\nThe basic simulation proposed here (Algo. 1) captures the key aspects of the problem and may\nguide the relevant genetics research. Yet, more detailed biophysical simulations represent a valuable\ndirection for future work. Our simulations suggest that the segmentation accuracy increases signi\ufb01-\ncantly with the inclusion of additional color channels, which coincides with ongoing experimental\nefforts [9, 10, 11]. We also note that color constancy of individual neurons plays an important role\nboth in the accuracy of the segmentation (Fig. 4) and the supervoxelized problem size.\nWhile we did not focus on post-processing in this paper, basic algorithms (e.g., reassignment of small,\nisolated supervoxels) may improve both the visualization and the segmentation quality. Similarly,\nmore elaborate formulations of the adjacency relationship between supervoxels can increase the\naccuracy. Finally, supervised learning of this relationship (when labeled data is present) is a promising\ndirection, and our methods can signi\ufb01cantly accelerate the generation of training sets.\n\n6 Acknowledgments\n\nThe authors thank Suraj Keshri and Min-hwan Oh (Columbia University) for useful conversations.\nFunding for this research was provided by ARO MURI W911NF-12-1-0594, DARPA N66001-\n15-C-4032 (SIMPLEX), and a Google Faculty Research award; in addition, this work was supported\nby the Intelligence Advanced Research Projects Activity (IARPA) via Department of Interior/ Interior\nBusiness Center (DoI/IBC) contract number D16PC00008. The U.S. Government is authorized to\nreproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation\nthereon. Disclaimer: The views and conclusions contained herein are those of the authors and should\nnot be interpreted as necessarily representing the of\ufb01cial policies or endorsements, either expressed\nor implied, of IARPA, DoI/IBC, or the U.S. Government.\n\n8\n\n\fReferences\n[1] J Livet et al. Transgenic strategies for combinatorial expression of \ufb02uorescent proteins in the nervous\n\nsystem. Nature, 450(7166):56\u201362, 2007.\n\n[2] A H Marblestone et al. Rosetta brains: A strategy for molecularly-annotated connectomics. arXiv preprint\n\narXiv:1404.5103, 2014.\n\n[3] Shin-ya Takemura, Arjun Bharioke, Zhiyuan Lu, Aljoscha Nern, Shiv Vitaladevuni, Patricia K Rivlin,\nWilliam T Katz, Donald J Olbris, Stephen M Plaza, Philip Winston, et al. A visual motion detection circuit\nsuggested by drosophila connectomics. Nature, 500(7461):175\u2013181, 2013.\n\n[4] D Cai et al. Improved tools for the brainbow toolbox. Nature methods, 10(6):540\u2013547, 2013.\n[5] F Chen et al. Expansion microscopy. Science, 347(6221):543\u2013548, 2015.\n[6] K Chung and K Deisseroth. Clarity for mapping the nervous system. Nat. methods, 10(6):508\u2013513, 2013.\n[7] E Betzig et al. Imaging intracellular \ufb02uorescent proteins at nanometer resolution. Science, 313(5793):1642\u2013\n\n1645, 2006.\n\n[8] M J Rust et al. Sub-diffraction-limit imaging by stochastic optical reconstruction microscopy (storm).\n\nNature methods, 3(10):793\u2013796, 2006.\n\n[9] A M Zador et al. Sequencing the connectome. PLoS Biol, 10(10):e1001411, 2012.\n[10] J H Lee et al. Highly multiplexed subcellular rna sequencing in situ. Science, 343(6177):1360\u20131363, 2014.\n[11] K H Chen et al.\nScience,\n\nSpatially resolved, highly multiplexed rna pro\ufb01ling in single cells.\n\n348(6233):aaa6090, 2015.\n\n[12] M Maggioni et al. Nonlocal transform-domain \ufb01lter for volumetric data denoising and reconstruction.\n\nImage Processing, IEEE Transactions on, 22(1):119\u2013133, 2013.\n\n[13] U Von Luxburg. A tutorial on spectral clustering. Statistics and computing, 17(4):395\u2013416, 2007.\n[14] D Lee and H S Seung. Algorithms for non-negative matrix factorization. In Advances in neural information\n\nprocessing systems, pages 556\u2013562, 2001.\n\n[15] F Meyer. Topographic distance and watershed lines. Signal processing, 38(1):113\u2013125, 1994.\n[16] F Meyer. Minimum spanning forests for morphological segmentation. In Mathematical morphology and\n\nits applications to image processing, pages 77\u201384. Springer, 1994.\n\n[17] J Cousty et al. Watershed cuts: Minimum spanning forests and the drop of water principle. Pattern Analysis\n\nand Machine Intelligence, IEEE Transactions on, 31(8):1362\u20131374, 2009.\n\n[18] Xiaofeng Ren and Jitendra Malik. Learning a classi\ufb01cation model for segmentation. In Computer Vision,\n\n2003. Proceedings. Ninth IEEE International Conference on, pages 10\u201317. IEEE, 2003.\n\n[19] J S Kim et al. Space-time wiring speci\ufb01city supports direction selectivity in the retina. Nature,\n\n509(7500):331\u2013336, 2014.\n\n[20] Gilles Bertrand and Gr\u00e9goire Malandain. A new characterization of three-dimensional simple points.\n\nPattern Recognition Letters, 15(2):169\u2013175, 1994.\n\n[21] Viren Jain, Benjamin Bollmann, Mark Richardson, Daniel R Berger, Moritz N Helmstaedter, Kevin L\nBriggman, Winfried Denk, Jared B Bowden, John M Mendenhall, Wickliffe C Abraham, et al. Boundary\nlearning by optimization with topological constraints. In Computer Vision and Pattern Recognition (CVPR),\n2010 IEEE Conference on, pages 2488\u20132495. IEEE, 2010.\n\n[22] J L Bentley. Multidimensional binary search trees used for associative searching. Communications of the\n\nACM, 18(9):509\u2013517, 1975.\n\n[23] Z Wu and R Leahy. An optimal graph theoretic approach to data clustering: Theory and its application to\n\nimage segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 15(11):1101\u20131113, 1993.\n\n[24] A Y Ng et al. On spectral clustering: Analysis and an algorithm. Advances in neural information processing\n\nsystems, 2:849\u2013856, 2002.\n\n[25] K Kurihara et al. Collapsed variational dirichlet process mixture models. In IJCAI, volume 7, pages\n\n2796\u20132801, 2007.\n\n[26] Jeffrey W Miller and Matthew T Harrison. A simple example of dirichlet process mixture inconsistency for\nthe number of components. In Advances in neural information processing systems, pages 199\u2013206, 2013.\n[27] Lawrence Hubert and Phipps Arabie. Comparing partitions. Journal of classi\ufb01cation, 2(1):193\u2013218, 1985.\n[28] U S\u00fcmb\u00fcl et al. A genetic and computational approach to structurally classify neuronal types. Nature\n\ncommunications, 5, 2014.\n\n[29] U S\u00fcmb\u00fcl et al. Automated computation of arbor densities: a step toward identifying neuronal cell types.\n\nFrontiers in neuroscience, 2014.\n\n[30] M H Longair et al. Simple neurite tracer: open source software for reconstruction, visualization and\n\nanalysis of neuronal processes. Bioinformatics, 27(17):2453\u20132454, 2011.\n\n[31] H Peng et al. V3d enables real-time 3d visualization and quantitative analysis of large-scale biological\n\nimage data sets. Nature biotechnology, 28(4):348\u2013353, 2010.\n\n9\n\n\f", "award": [], "sourceid": 1048, "authors": [{"given_name": "Uygar", "family_name": "S\u00fcmb\u00fcl", "institution": "Columbia University"}, {"given_name": "Douglas", "family_name": "Roossien", "institution": "University of Michigan"}, {"given_name": "Dawen", "family_name": "Cai", "institution": "University of Michigan"}, {"given_name": "Fei", "family_name": "Chen", "institution": "Massachusetts Institute of Technology"}, {"given_name": "Nicholas", "family_name": "Barry", "institution": "Massachusetts Institute of Technology"}, {"given_name": "John", "family_name": "Cunningham", "institution": "University of Columbia"}, {"given_name": "Edward", "family_name": "Boyden", "institution": "Massachusetts Institute of Technology"}, {"given_name": "Liam", "family_name": "Paninski", "institution": "Columbia University"}]}