{"title": "Elliptical Perturbations for Differential Privacy", "book": "Advances in Neural Information Processing Systems", "page_first": 10185, "page_last": 10196, "abstract": "We study elliptical distributions in locally convex vector spaces, and determine conditions when they can or cannot be used to satisfy differential privacy (DP). A requisite condition for a sanitized statistical summary to satisfy DP is that the corresponding privacy mechanism must induce equivalent probability measures for all possible input databases. We show that elliptical distributions with the same dispersion operator, $C$, are equivalent if the difference of their means lies in the Cameron-Martin space of $C$. In the case of releasing finite-dimensional summaries using elliptical perturbations, we show that the privacy parameter $\\ep$ can be computed in terms of a one-dimensional maximization problem. We apply this result to consider multivariate Laplace, $t$, Gaussian, and $K$-norm noise. Surprisingly, we show that the multivariate Laplace noise does not achieve $\\ep$-DP in any dimension greater than one. Finally, we show that when the dimension of the space is infinite, no elliptical distribution can be used to give $\\ep$-DP; only $(\\epsilon,\\delta)$-DP is possible.", "full_text": "Elliptical Perturbations for Differential Privacy\n\nMatthew Reimherr \u2217\nDepartment of Statistics\n\nPennsylvania State University\n\nUniversity Park, PA 16802\n\nmreimherr@psu.edu\n\nDepartment of Statistics\n\nPennsylvania State University\n\nUniversity Park, PA 16802\n\nJordan Awan \u2020\n\nawan@psu.edu\n\nAbstract\n\nWe study elliptical distributions in locally convex vector spaces, and determine\nconditions when they can or cannot be used to satisfy differential privacy (DP).\nA requisite condition for a sanitized statistical summary to satisfy DP is that the\ncorresponding privacy mechanism must induce equivalent probability measures\nfor all possible input databases. We show that elliptical distributions with the\nsame dispersion operator, C, are equivalent if the difference of their means lies\nin the Cameron-Martin space of C. In the case of releasing \ufb01nite-dimensional\nsummaries using elliptical perturbations, we show that the privacy parameter \u0001\ncan be computed in terms of a one-dimensional maximization problem. We apply\nthis result to consider multivariate Laplace, t, Gaussian, and K-norm noise. Sur-\nprisingly, we show that the multivariate Laplace noise does not achieve \u0001-DP in\nany dimension greater than one. Finally, we show that when the dimension of the\nspace is in\ufb01nite, no elliptical distribution can be used to give \u0001-DP; only (\u0001, \u03b4)-DP\nis possible.\n\n1\n\nIntroduction\n\nIn\ufb01nite dimensional objects and parameters arise commonly in nonparametric statistics, shape anal-\nysis, and functional data analysis. Several recent works have made strides towards extending tools\nfor differential privacy (DP) to handle such settings. Some of the \ufb01rst results in this area were given\nin Hall et al. (2013), with a particular emphasis on Gaussian perturbations and point-wise releases of\nstatistical summaries represented as univariate functions. This work was extended to more general\nBanach and Hilbert space based summaries by Mirshani et al. (2017), which included protections for\npublic releases based on path level summaries, nonlinear transformations of functional summaries,\nand full function releases as well. However, Gaussian perturbations are not always satisfactory since\nthey cannot be used to achieve pure DP (\u0001-DP), which requires heavier tailed distributions. Rather,\nfor pure DP, the most popular distribution is the Laplace mechanism, whose tails are \u201cjust right\" for\nachieving DP in \ufb01nite dimensional summaries (Dwork et al., 2006).\nWhen one moves from univariate to multivariate settings, generalizing the Laplace mechanism is\nnot as simple as generalizing the Gaussian. Often, when the Laplace mechanism is used in mul-\ntivariate settings, iid Laplace random variables are used. However, this approach fails to capture\nthe multivariate dependence structure of the data or parameter of interest. Furthermore, in in\ufb01nite\ndimensional settings, adding iid noise is usually not an option if one wishes to remain in a particular\nfunction space. To address these issues, we study the use of elliptical distributions to satisfy DP,\nwhich allow for a dispersion operator and are closely related to Gaussian distributions. Elliptical\n\u2217Research supported in part by NSF DMS 1712826, NSF SES 1853209, and the Simons Institute for the\n\u2020Research supported in part by NSF SES-153443 and NSF SES-1853209.\n\nTheory of Computing at UC Berkeley.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fprocesses offer a nice option for designing DP mechanisms for multivariate and in\ufb01nite-dimensional\ndata as they allow for the customization of both the tail behavior and dependence structure, which\ncan be tailored to the problem at hand. Recently Bun and Steinke (2019) explored several alterna-\ntive univariate distributions for achieving privacy such as Cauchy, Student\u2019s T, Laplace Log-Normal,\nUniform Log-Normal, and Arsinh-Normal, which can be extended to elliptical distributions.\nWe are interested in releasing a sanitized version of a statistic T : D \u2192 X, where D is a metric\nspace, representing the space of possible input databases, D, and X is a locally convex vector space.\n\nTo achieve differential privacy, we will release (cid:101)T = T (D) + \u03c3X, where \u03c3 is a positive scalar, and X\n\nis a random element of X. In particular, we consider X which are drawn from elliptical distributions,\nof which the multivariate Laplace and Gaussian distributions are special cases. Most linear spaces\nare locally convex vector spaces, including all Hilbert Spaces, Banach Spaces, Frechet spaces, and\nproduct spaces of normed vector spaces, meaning that our results will hold quite broadly.\nWe consider the setting where the statistical summary and privacy mechanism are truly in\ufb01nite\ndimensional, meaning that the problem cannot be embedded into a \ufb01nite dimensional subspace\nwhere multivariate privacy tools can be used. There are both interesting mathematical and practical\nmotivations for this perspective. First, our setting can be viewed as a limit of multivariate problems;\nif one has privacy over the full in\ufb01nite dimensional space, then this ensures that the noise is well\nbehaved when releasing multivariate summaries, regardless of how many are released. Second, one\ndoes not need to ensure that every database uses the same \ufb01nite dimensional subspace, allowing\npractitioners to use whatever methods and summaries they prefer. And third, our setting is very\nconvenient when addressing multiple queries. In particular, one does not need to spend a fraction of\nthe privacy budget for every query. Instead, the amount spent for each subsequent query decreases\ndramatically, eventually leveling out to a maximum \u0001 or (\u0001, \u03b4). To accomplish this, one does not\nneed to \u201cstore\u201d the in\ufb01nite dimensional noise, instead, we can generate as much of the noise as is\nneeded for a particular query while conditioning on any noise values generated for prior queries.\nWe also provide a surprising result showing that \u0001-DP can only be achieved for a \ufb01nite number of\nsummaries or point-wise evaluations; in in\ufb01nite dimensions no elliptical perturbation is capable of\nachieving \u0001-DP over the full function space, one can only achieve (\u0001, \u03b4)-DP. This is in stark contrast\nwith what is known from the univariate or multivariate literature on DP.\nWhile elliptical distributions are being used more frequently in statistics and machine learning (e.g.\nSchmidt, 2003; Frahm et al., 2003; Soloveychik and Wiesel, 2014; Couillet et al., 2016; Sun et al.,\n2016; Goes et al., 2017; Ollila and Raninen, 2019), some fundamental questions regarding elliptical\ndistributions in function spaces remain underdeveloped. For data privacy, the question of equiva-\nlence/orthogonality of elliptical measures is particularly important. In terms of data privacy, if a\nperturbation in a dataset produces a private summary that is orthogonal (in a probabilistic sense) to\nthe old one, then the summaries cannot be differentialy private since they can be distinguished with\nprobability one. We show that several conditions for making this determination transfer nicely from\nthe Gaussian setting, but not all. While conditions on the location function remain the same, con-\nditions on the dispersion function change. Furthermore, that all elliptical measures are equivalent\nor orthogonal need no longer hold without additional assumptions. Regardless, for the purposes of\nprivacy, determining equivalence/orthogonality based on the location is the primary requirement.\nRelated Work: Our general approach of adding noise from a data-independent distribution to a\nsummary statistic is one of the simplest and most common methods of achieving DP. This approach\nwas \ufb01rst developed using the Laplace mechanism (Dwork et al., 2006), and has since been expanded\nto include a larger variety of distributions. Ghosh et al. (2009) showed that when the data is a count,\nthen the optimal noise adding distribution is discrete Laplace. Geng and Viswanath (2016) extended\nthis result to the continuous setting, developing the staircase distribution which is closely related to\ndiscrete Laplace. In the multivariate setting, the most common solution is to add iid Laplace noise\nto each coordinate (Dwork and Roth, 2014). However, Hardt and Talwar (2010) and Awan and\nSlavkovi\u00b4c (2019) demonstrate that capturing the covariance structure in the data, via the K-norm\nmechanism can substantially reduce the amount of noise required.\nAfter adding noise to summary statistics, researchers have shown that many complex statistical and\nmachine learning tasks can be produced by post-processing, such as linear regression (Zhang et al.,\n2012), maximum likelihood estimation (Karwa and Slavkovi\u00b4c, 2016), hypothesis testing (Vu and\nSlavkovi\u00b4c, 2009; Gaboardi et al., 2016; Awan and Slavkovi\u00b4c, 2018), posterior inference (Williams\nand Mcsherry, 2010; Karwa et al., 2016), or general asymptotic analysis (Wang et al., 2018).\n\n2\n\n\fTo date, the only additive mechanism in in\ufb01nite-dimensions is the Gaussian mechanism, developed\nby Hall et al. (2013) and Mirshani et al. (2017). However, there has been other work on developing\nprivacy tools for these spaces. Awan et al. (2019) show that the exponential mechanism (McSherry\nand Talwar, 2007) can be used in arbitrary Hilbert spaces, by integrating with respect to a \ufb01xed\nprobability measure such as a Gaussian process. An alternative approach proposed by Alda and\nRubinstein (2017) uses Bernstein polynomial approximations to release private functions. Recently,\nSmith et al. (2018) utilized the techniques of Hall et al. (2013) to develop private Gaussian process\nregression. Similar to the puffer\ufb01sh approach (Kifer and Machanavajjhala, 2014), they assume that\nthe predictors are public, and use the known covariance structure to tailor the noise distribution.\nOrganization: In Section 2, we review the necessary background on locally convex vector spaces,\nelliptical distributions, and differential privacy. In Section 3, we study the equivalence and orthog-\nonality of elliptical measures, and give a condition that ensures that two elliptical measures are\nequivalent. In Section 4, we investigate using elliptical perturbations to achieve DP. First, we con-\nsider the \ufb01nite dimensional case in Section 4.1, and in Theorem 3 we give a condition for elliptical\nperturbations to satisfy \u0001-DP as well as a method of computing \u0001. In Section 4.2 we show that if the\ndimension of the space is in\ufb01nite, then no elliptical perturbation can satisfy \u0001-DP. In fact, we show\nthat every elliptical distribution can only achieve (\u0001, \u03b4)-DP for a positive \u03b4. We give short proof\nsketches throughout the document, with detailed proofs left to the Supplementary Material.\n\n2 Elliptical Distributions\nElliptical distributions, whether over Rd or a more general vector space can be de\ufb01ned in a variety\nof equivalent ways. Intuitively, an elliptical distribution is one in which its density contours form\nhyperellipses. However, this presupposes that the measure is absolutely continuous with respect to\nLebesgue measure. Thus, it is often useful in multivariate settings to use alternative de\ufb01nitions that\nare more easy to generalize, but which are equivalent to the shape of the density contours when they\nexist. This is not unique to elliptical measures, such alternative de\ufb01nitions are often useful when\nworking with in\ufb01nite dimensional objects (e.g. Bosq, 2000). Throughout, we focus our attention on\nan arbitrary, real, locally convex vector space (from here on LCS), X, but we will restrict ourselves\nto simpler spaces (e.g. Banach, Hilbert, or Euclidean spaces) as needed or for illustration. For\nease of reference, recall the following concepts from functional analysis (see Rudin (1991) for an\nintroduction).\n\nthose operations are well de\ufb01ned).\n\n\u2022 A set, X, is called a vector space if it is closed under addition and scalar multiplication (and\n\u2022 A vector space, X, is called a topological vector space, if it is equipped with a topology\nunder which addition and scalar multiplication are continuous.\n\u2022 A topological vector space X is called locally convex if its topology is generated by a\nseparated family of semi-norms, {p\u03b1 : \u03b1 \u2208 I}, where I is an arbitrary index set and\nseparated means that for all nonzero x \u2208 X there exists \u03b1 \u2208 I such that p\u03b1(x) (cid:54)= 0. A base\nfor the topology is given by sets of the form A\u03b1,\u0001 = {x \u2208 X : p\u03b1(x) < \u0001}.\n\n\u2022 The topological dual, X\u2217, is the collection of all continuous linear functionals on X.\n\nThe assumption that the seminorms are separated is not always included in the de\ufb01nition, but is\nequivalent to assuming that the space is Hausdorff. Recall that a topology de\ufb01nes the open sets, a\ncollection of subsets that is closed under uncountable unions, \ufb01nite intersections, and contains both\nX and \u2205. We use this level of generality to include as many settings as possible into our framework.\nIn particular, all \ufb01nite dimensional Euclidean spaces, normed vector spaces, Hilbert Spaces, Banach\nSpaces, and Frechet spaces are types of LCS. In addition, uncountable product spaces of normed\nspaces, which are often used in the mathematical foundations of stochastic processes, are LCS as\nwell (when equipped with the product topology). To \ufb01nd practical examples of spaces that are not\nlocally convex spaces, one either has to consider nonlinear spaces, such as manifolds, or equip a\nspace with an \u201codd\" metric (such as Lp for p < 1).\nExample 1 (LCS Examples). The de\ufb01nition of LCS in terms of seminorms is perhaps unintuitive\nat \ufb01rst, but can be motivated by product spaces such as R[0,1] = {f : [0, 1] \u2192 R}. The space R[0,1]\nis in a sense \u201ctoo large\u201d to accommodate a norm, but it is easy to de\ufb01ne a family of semi-norms\nthat measure the magnitude of the function coordinate-wise. Here \u03b1 \u2208 I := [0, 1] and we de\ufb01ne\n\n3\n\n\fp\u03b1(f ) = |f (\u03b1)|. Note that for any particular \u03b1, p\u03b1 is not a norm, since p\u03b1(f\u2212g) = 0 does not imply\nthat f = g. However, the entire collection of semi-norms separates points since p\u03b1(f \u2212 g) = 0 for\nall \u03b1 \u2208 [0, 1] implies that f = g.\nIt is easy to see that any normed space \ufb01ts the de\ufb01nition of LCS. For C[0, 1], we set I = {1} and\n\nde\ufb01ne p1(f ) = supt\u2208[0,1] |f (t)|, and for L2[0, 1] we set I = {1} and de\ufb01ne p1(f ) =(cid:82) f 2(t)dt.\n\nWhen working with a LCS one commonly uses one of two \u03c3-algebras. The Borel \u03c3-algebra, B, is the\nsmallest \u03c3-algebra that contains the open sets. The cylinder \u03c3-algebra, C, is the smallest \u03c3-algebra\nthat makes all continuous linear functionals measurable. In general we have C \u2286 B, but these two\n\u03c3-algebras are not equal unless the space has additional structure, e.g. separable Banach spaces.\nThis creates complications in in\ufb01nite dimensional settings. For example, the technical theory for\nstochastic processes often starts with product spaces such as R[0,1]. There, the two sigma algebras\nare not the same, which is an issue for privacy as one desires privacy over B, not just C. This\nis because only the events in the chosen \u03c3-algebra are protected, and a larger \u03c3-algebra offers a\nstronger privacy guarantee. More importantly, C does not contain most sets of interest, including\ncontinuous functions, linear functions, polynomials, constant functions, etc.\n(Billingsley, 1979,\nProblem 36.6). To overcome this challenge, Mirshani et al. (2017) used Cameron-Martin theory, to\nobtain DP over all of B through careful use of densities in in\ufb01nite dimensional spaces. This theory\nis built upon Gaussian processes; however, we will show that several of their key results, especially\nthose needed for privacy, extend directly to elliptical distributions. Throughout this paper, we assume\nX is equipped with its Borel \u03c3-algebra when discussing measures, measurability, and DP.\nOften it is convenient to de\ufb01ne probability measures over abstract spaces in terms of their character-\nistic functionals (i.e. Fourier transforms), which uniquely determine measures in any LCS.\nDe\ufb01nition 1 (Fang, 2017). A measure, P , over a locally convex space X is called elliptical if and\n\nonly if its characteristic functional, (cid:101)P : X\u2217 \u2192 C, has the form\n\nexp{ig(x)} dP (x) = eig(\u00b5)\u03c60(C(g, g)),\n\n(cid:101)P (g) =\n\n(cid:90)\n\nX\n\nwhere \u00b5 \u2208 X, C is a symmetric, positive de\ufb01nite, continuous bilinear form on X\u2217 \u00d7 X\u2217, and \u03c60 is a\npositive de\ufb01nite function over R, which is continuous at 0 and satis\ufb01es \u03c60(0) = 1.\nDe\ufb01nition 1 implies that the distribution of P is uniquely determined by knowing \u00b5, C, and \u03c60. The\nobject \u00b5 denotes the center of the distribution; we will say a distribution is centered if \u00b5 = 0. The\nobject C is often called the covariance or dispersion operator. In general, C can either be identi\ufb01ed\nas an operator or bilinear form (in fact a (0, 2) tensor). We avoid introducing extra notation we will\nlet C(g) denote the operator version and C(f, g) the bilinear form.\nWe begin by presenting a second characterization of elliptical measures. This is a well known result,\nbut we are unaware of a reference for this level of generality. We cite Fang (2017), which covers the\nmultivariate case, but the proof is the same for general LCS.\nTheorem 1 (Fang, 2017). Let X \u2208 X be an elliptically distributed random variable. Then there\nexists a mean zero Gaussian process Z \u2208 X with covariance operator C, an element \u00b5 \u2208 X, and a\nstrictly positive random variable V \u2208 R+, that is independent of Z, and satis\ufb01es X\nThis result is often phrased as \u201cevery elliptical distribution is a scalar mixture of Gaussian pro-\ncesses.\" While it is, of course, a fascinating result in its own right, it also provides a simple method\nof generating and simulating from arbitrary elliptical distributions.\nDue to this corollary, we will index every elliptical measure using \u00b5, C, and the mixing distribution\nof V , which we will denote as \u03c8, and use the notation E(\u00b5, C, \u03c8). Equivalently, we could index\nusing the \u03c60 from De\ufb01nition 1, but our results in Section 3 are easier to present in terms of \u03c8.\nWe conclude by stating a general de\ufb01nition of DP, which makes sense over any measurable space,\nthough we state it here for LCS. The concept of differential privacy was \ufb01rst introduced in Dwork\net al. (2006) and Dwork (2006). Over time researchers have worked to make the de\ufb01nition more\nprecise and \ufb02exible, such as Wasserman and Zhou (2010) who state it in terms of conditional distri-\nbutions. For a general, axiomatic treatment of formal privacy, see Kifer et al. (2012).\nDe\ufb01nition 2. Let (D, d) be a metric space and {PD : D \u2208 D} be a family of probability measures\nover a locally convex topological vector space X. We say the family achieves (\u0001, \u03b4)-differential\n\nL\n= \u00b5 + V Z.\n\n4\n\n\fprivacy if for any d(D, D(cid:48)) \u2264 1 and any measurable A, we have\nPD(A) \u2264 e\u0001PD(cid:48)(A) + \u03b4.\n\n(1)\nIntuitively, D represents the universe of possible input databases. One then refers to {PD : D \u2208 D}\nas the privacy mechanism. The most common setting when discussing DP is when D is a product\nspace and the metric is the Hamming distance. However, the Hamming distance (which counts\ndifferences in coordinates) is insensitive to the magnitude of the difference between two inputs D\nand D(cid:48), thus one may wish to consider alternatives and so we take the more general approach. As\ndiscussed earlier, while DP can be de\ufb01ned with any \u03c3-algebra, we assume that X is equipped with\nthe Borel \u03c3-algebra as it offers more intuitive guarantees. We refer to (\u0001, \u03b4)-DP as approximate DP\nwhen \u03b4 > 0 and as pure-DP when \u03b4 = 0. When using pure DP, we often just write \u0001-DP.\nAnother way of viewing \u0001-DP (that is, taking \u03b4 = 0) is through the equivalence/orthogonality of\nprobability measures. As was discussed in Awan et al. (2019), in an \u0001-DP mechanism the individual\nmeasures that make up the mechanism are all equivalent in a probabilistic sense (meaning they agree\non the zero sets). Conversely, if the measures are orthogonal then the mechanism cannot even be\n(\u0001, \u03b4)-DP. This perspective was used in Mirshani et al. (2017) for the case of Gaussian mechanisms.\nHowever, the corresponding theory for elliptical distributions is less developed. In the next section\nwe extend several fundamental results of Gaussian processes to elliptical distributions.\n\n3 Equivalence and Orthogonality of Elliptical Measures\n\nA classic result from probability theory is that any two Gaussian processes are either equivalent or\northogonal (that is, as probability measures they either agree on the zero sets or concentrate their\nmass on disjoint sets). Recall that by the Radon-Nikodym theorem, if two measures are equivalent\nthen there exists a density of one with respect to the other (and vice versa). What we will now show\nis that this property, to a degree, extends to any elliptical family. Furthermore, we will show that\nthe conditions for establishing this equivalence/orthogonality are nearly the same as for Gaussian\nprocesses. We begin with a fairly simple yet surprisingly useful technical lemma.\nLemma 1. Let (\u2126,F, P ) be a probability space. Let X 1 : \u2126 \u2192 X and X 2 : \u2126 \u2192 X denote two\nrandom elements of X, and let T 1 : \u2126 \u2192 R and T 2 : \u2126 \u2192 R be two random variables. Let P i\ndenote the probability measure over X induced by X i and let Qi denote the measure over R induced\nt denote the conditional measure of X i given T i = t. If Q1 and Q2 are equivalent and\nby T i. Let P i\nt and P 2\nP 1\n\nt are equivalent for almost all t (wrt Q1) then so are P 1 and P 2.\n\n0 and P 1\n\n0 = P 1\n\n1 are orthogonal and set P 2\n\nImplicit in Lemma 1 is that the con-\nThe proof of Lemma 1 is in the Supplementary Material.\nditional distributions exist. This is not an issue in our setting as the conditional distributions can\nbe explicitly constructed for elliptical processes, however, for general processes and spaces one\ncan encounter nontrivial technical problems. We refer the interested reader to Hoffmann-J\u00f8rgensen\n(1972), Bogachev (1998, THM A.3.11), and Kallenberg (2006, Chapter 6) for further discussion.\nInterestingly, the reverse statement is not true. That is, even if all of the conditional distributions\nare orthogonal, the unconditional measures need not be orthogonal. To see this, suppose that T is\n1 and\n0 or 1 with equal probability. Now, assume that P 1\n0 . Clearly the conditional distributions are orthogonal, but not only are the unconditional\n1 = P 1\nP 2\nmeasures equivalent, they are actually the same!\nRegardless, our goal is more speci\ufb01c; we want to establish conditions under which E(\u00b51, C, \u03c8) and\nE(\u00b52, C, \u03c8) are orthogonal when they share the same \u03c8 and C. In terms of DP, \u00b51 and \u00b52 represent\nthe private summary from two different databases. If \u03c8 is a point mass, then the two measures are\nGaussian and the conditions are known. The question is, to what degree do such conditions extend\nto other mixtures? Theorem 2 shows that the same conditions for Gaussian processes (with the same\ncovariance, but different means) apply to any elliptical family. Given Corollary 1, this may seem\nobvious, but Lemma 1 implies that the matter is surprisingly delicate. For example, two Gaussian\nprocesses with the same mean, but where one has a covariance equal to a scalar c (cid:54)= 1 multiple of\nthe other, are actually orthogonal (in in\ufb01nite dimensions). This need not hold for arbitrary elliptical\nfamilies as the scalar can be absorbed by the mixing coef\ufb01cient (and then apply Lemma 1).\nOur \ufb01rst major result establishes a condition under which DP cannot be achieved, regardless of\nthe magnitude of the noise. First, let us de\ufb01ne a subspace of X using the bilinear form C (more\n\n5\n\n\fproduct (cid:104)\u00b7,\u00b7(cid:105)K on the dual space X\u2217 given by (cid:104)f, g(cid:105)K := C(f, g) = (cid:82) f (x)g(x)dP (x), where P\n\ndetail can be found in Bogachev (1998); Mirshani et al. (2017)). In particular, C induces an inner\nis a Gaussian measure with mean zero and covariance C. Then, we can view X\u2217 as a subspace of\nL2(X, P ), the space of P -square integrable functions from X \u2192 R. By assumption, (cid:104)\u00b7,\u00b7(cid:105)K is a\ncontinuous, symmetric, and positive de\ufb01nite bilinear form and thus a valid inner product. However,\nX\u2217 is not complete with respect to this inner product when X is in\ufb01nite dimensional, so let K denote\nits completion. Finally, consider the subset H \u2282 X, such that for h \u2208 H the operation Th : K \u2192 R\ngiven by Th(g) := g(h) is continuous in the K topology. Then H is called the Cameron-Martin\nspace of C (or equivalently, of the mean zero Gaussian process with C as its covariance). Intuitively,\nthe functionals in K are much \u201crougher\" than those in X\u2217 and thus the elements of H are much more\nregular than general elements of X to counter balance this. In fact, C also generates an operator\nexactly when it equals h = C(g) for some g \u2208 K. The space H is also a Hilbert space (even though\nX need not be) equipped with the inner product (cid:104)h1, h2(cid:105)H = (cid:104)g1, g2(cid:105)K where hi = C(gi).\nTheorem 2. Let P1 \u223c E(\u00b51, C, \u03c8) and P2 \u223c E(\u00b52, C, \u03c8) be two elliptical measures over a locally\nconvex topological vector space, X. Then the two distributions are equivalent if \u00b51 \u2212 \u00b52 resides in\nthe Cameron-Martin space of C and orthogonal otherwise.\n\nfrom K \u2192 H denoted as C(g) =(cid:82) xg(x)dP (x). Using this notation, an element h \u2208 X lies in H\n\nProof Sketch. For the \ufb01rst direction, if \u00b51 \u2212 \u00b52 resides in the Cameron-Martin space of C then it\nresides in the Cameron-Martin space of vC for v > 0 since they induce equivalent norms. From\nBogachev (1998, Theorem 2.4.5 ), two Gaussian measures with the same covariance, C, are equiv-\nalent if the difference of their means resides in the Cameron-Martin space of C. Thus, conditioned\non the mixture V = v, the measures are equivalent for all v. By Lemma 1, they are equivalent.\nFor the reverse direction we consider, without loss of generality, X1 \u223c E(0, C, \u03c8) versus X2 \u223c\nE(\u00b5, C, \u03c8) where \u00b5 is not in the Cameron-Martin space of C. To see that the two measures are\northogonal, it suf\ufb01ces to show that, for any \ufb01xed \u0001 \u2208 (0, 1) we can construct a measurable set A\nsuch that P (X1 \u2208 A) \u2265 1 \u2212 \u0001 while P (X2 \u2208 A) \u2264 \u0001.\n\nTo interpret Theorem 2 in the context of privacy, given a database D \u2208 D, recall that a private\nsummary is drawn from the elliptical distribution E(\u00b5D, C, \u03c8). Theorem 2 then says that the mea-\nsures are orthogonal (and thus no amount of noise will produce a DP summary) unless all of the\ndifferences \u00b5D \u2212 \u00b5D(cid:48), for any D, D(cid:48) \u2208 D reside in the Cameron-Martin space of C.\n\n4 Achieving DP with Elliptical Perturbations\n\nNow that we have the necessary tools in place and we know when we cannot have DP, we will now\nconstruct a broad class of mechanisms that do achieve DP. Recall that the mechanisms will be of the\n\nform (cid:101)TD = TD + \u03c3X, where TD := T (D) is the nonprivate statistical summary, X is a prespeci\ufb01ed\n\nelliptical process and \u03c3 > 0 is a \ufb01xed scalar. The exact value of \u03c3 will be set to achieve some desired\nlevel of privacy. Gaussian perturbations (i.e. taking \u03c6 as a point mass) will not achieve \u0001-DP even in\n\ufb01nite dimensions. As is known in the literature, Gaussian perturbations have tails that are too light,\ncausing the probability inequality of DP to fail for sets in the tails. To \ufb01x this, it is common to use\nanother distribution, often the Laplace distribution, whose tails appear to be just right for achieving\nDP. Interestingly, this trick does not carry over to in\ufb01nite dimensional spaces. We will show that\nwhile some elliptical distributions can achieve \u0001-DP for \ufb01nite dimensional projections, none can\nachieve it over the entire in\ufb01nite-dimensional space; they can only achieve (\u0001, \u03b4)-DP with \u03b4 > 0.\n\n4.1 DP in Finite Dimensions\n\nIn this subsection, we give a criterion (Theorem 3) that establishes which elliptical distributions\nsatisfy \u0001-DP, when X = Rd. We also provide a related result (Corollary 1) for \u0001-DP with d-\ndimensional projections of in\ufb01nite dimensional summaries, which holds uniformly across the choice\nof projection, for a \ufb01xed d. Elliptical distributions that can achieve \u0001-DP (with a \ufb01xed d) include (cid:96)2-\nmechanism (Chaudhuri and Monteleoni, 2009; Chaudhuri et al., 2011; Kifer et al., 2012; Song et al.,\n2013; Yu et al., 2014; Awan and Slavkovi\u00b4c, 2019), and the multivariate t distribution. Interestingly,\nthe multivariate Laplace distribution cannot achieve \u0001-DP when d \u2265 2.\n\n6\n\n\fDenote by \u03a3 = {C(ei, ej)} the positive de\ufb01nite matrix containing the evaluations of C on the\n\nstandard basis of Rd. Then the density of(cid:101)TD = TD +\u03c3X is proportional to f (\u03c3\u22122(x\u2212TD)\u03a3\u22121(x\u2212\nTD)), where f is a decreasing positive function depending only on the dimension d and the elliptical\nfamily for X. The omitted constants depend on \u03a3, but not on TD. The Cameron-Martin norm can\nTheorem 3. Assume that X = Rd and, without loss of generality, assume that that (cid:101)TD has a density\nbe expressed as (cid:107)g(cid:107)H = g(cid:62)\u03a3\u22121g. In fact H = Rd, but equipped with a different norm.\nwith respect to Lebesgue measure proportional to f(cid:101)TD\nf : [0,\u221e) \u2192 [0,\u221e] is a decreasing positive function. Set\n(cid:107)TD \u2212 TD(cid:48)(cid:107)H = sup\nD\u223cD(cid:48)\nf ((c \u2212 \u2206)2)\n\n(x) \u221d f (\u03c3\u22122(x\u2212TD)(cid:62)\u03a3\u22121(x\u2212TD)), where\n\nIf \u2206 < \u221e, f (0) < \u221e, and\n\n(cid:107)\u03a3\u22121/2(TD \u2212 TD(cid:48))(cid:107)2.\n\n\u2206 = sup\nD\u223cD(cid:48)\n\n(2)\n\nthen (cid:101)TD satis\ufb01es \u0001-DP, where exp(\u0001) = sup\n\nlim sup\nc\u2192\u221e\n\n< \u221e,\n\nf (c2)\n\nf ((c \u2212 \u03c3\u22122\u2206)2)\n\n< \u221e.\n\nc\u2265\u03c3\u22121\u2206\n\nf (c2)\n\n(cid:17)\n\ni=1 |xi \u2212 \u00b5i|/\u03c3i\n\n(cid:16)\u2212(cid:80)d\n\nthis mechanism is proportional\n\nfor achieving \u0001-DP. The density of\n\nThe proof of Theorem 3 is based on the ratio of the densities, and is in the Supplementary Materials.\nNext we apply Theorem 3 to several common distributions.\nExample 2 (Independent Laplace).\nIndependent Laplace random variables are a common\nto f (x) \u221d\ntool\n. While it is easily proved that this mechanism can be used to sat-\nexp\nisfy \u0001-DP, this distribution is not elliptical, since the density cannot be written as a function of\n(x \u2212 \u00b5)(cid:62)\u03a3\u22121(x \u2212 \u00b5) for any \u00b5 and \u03a3.\nA natural idea is to use the elliptical multivariate Laplace distribution to try to achieve \u0001-DP for\nmulti-dimensional outputs. Surprisingly, the following example shows that while the tail behavior\nof the multivariate Laplace is suf\ufb01cient to satisfy (2), the multivariate Laplace distribution cannot be\nused to achieve \u0001-DP when d \u2265 2, since it has a pole (i.e. goes to in\ufb01nity) at its center.\nExample 3 (Multivariate Laplace). A d-dimensional random variable X \u223c Laplace(\u00b5, \u03a3) has\ndensity equal to\n\n2(2\u03c0)\u2212d/2|\u03a3|\u22121/2(cid:0)(x \u2212 \u00b5)(cid:62)\u03a3\u22121(x \u2212 \u00b5)/2(cid:1)\u03bd/2\n\n2(x \u2212 \u00b5)(cid:62)\u03a3\u22121(x \u2212 \u00b5)),\n\n(cid:113)\n\nwhere \u03bd = 2\u2212d\n\ntional to f ((x \u2212 \u00b5)(cid:62)\u03a3\u22121(x \u2212 \u00b5)), where f (y) = (y/2)\u03bd/2 K\u03bd((cid:112)2y). The reason this distribution\n\nand K\u03bd is the modi\ufb01ed Bessel function of the second kind. This density is propor-\n\nis called the multivariate Laplace distribution is that it is the only family of distributions such that\nevery marginal distribution is also distributed as Laplace (iid Laplace does not have this property).\nFirst, let\u2019s check whether (2) is \ufb01nite. We use the fact that K\u03bd(z) = c exp(\u2212z)z\u22121/2(1 + O(1/z))\nas z \u2192 \u221e, where c is a constant (Abramowitz and Stegun, 1965, Chapter 9). Then\n\u221a\n2(c \u2212 \u2206))\nc\nc \u2212 \u2206\n2c)\n\nlim\nc\u2192\u221e\nWe see that the tails of the multivariate Laplace distribution are heavy enough to satisfy \u0001-DP. How-\never, it turns out that there is another problem in this case, which is that f (x) has a pole at x = 0.\n\nWe use the fact that for 0 < x (cid:28)(cid:112)|\u03bd| + 1, as x \u2192 0+, K\u03bd(x) is asymptotically similar to\n\n\u221a\n2(c \u2212 \u2206))\n\u221a\nK\u03bd(\nK\u03bd(\n2c)\n\n(cid:1)\u03bd\n(cid:0) c\u2212\u2206\n(cid:0) c\n(cid:1)\u03bd\n\nexp(\u2212\u221a\nexp(\u2212\u221a\n\n\u221a\n= exp(\n\nf ((c \u2212 \u2206)2)\n\n= lim\nc\u2192\u221e\n\n= lim\nc\u2192\u221e\n\nf (c2)\n\n\u221a\n\nK\u03bd(\n\n2\n\n2\n\n2\n\n2\u2206).\n\nFrom this, we see that the limit is \ufb01nite when d = 1, but in\ufb01nite when d \u2265 2. So, the multivariate\nLaplace distribution cannot be used to achieve \u0001-DP for d \u2265 2.\n\n7\n\nwhere \u03b3 is a constant (Abramowitz and Stegun, 1965, Chapter 9). Then\n\nK\u03bd(x) \u223c\n\n(cid:40)\u2212 log(x)\nK\u03bd((cid:112)2y) \u221d lim\n\n\u0393(\u03bd)\n\n2 (2/x)|\u03bd|\n\ny\u21920+\n\n(cid:16) y\n\n(cid:17)\u03bd/2\n\nif \u03bd = 0\nif \u03bd (cid:54)= 0,\n\n\uf8f1\uf8f4\uf8f2\uf8f4\uf8f3exp(\u2212\u221a\n\ny)\n\u2212 1\n2 log(2y)\n(y/2)\u03bd/2( 2\u221a\n\nif d = 1\nif d = 2\nif d \u2265 3\n\n2y )|\u03bd|\n\nlim\ny\u21920+\n\nf (y) =\n\n2\n\n\fWhile we may have supposed that the multivariate Laplace distribution would be well suited for \u0001-\nDP, in fact it seems that the K-norm mechanism, introduced by Hardt and Talwar (2010), is a better\ngeneralization of the Laplace mechanism, since it is carefully tuned for privacy.\nExample 4 (K-Norm Mechanism). For any norm (cid:107)\u00b7(cid:107)K, the K-norm mechanism with mean \u00b5 draws\nfrom the density proportional to exp(\u2212(cid:107)x\u2212\u00b5(cid:107)K). For norms of the form (cid:107)x(cid:107) =\nx(cid:62)\u03a3\u22121x, the K-\nnorm mechanism is an elliptical distribution, with density is proportional to f ((x\u2212\u00b5)(cid:62)\u03a3\u22121(x\u2212\u00b5)),\nwhere f (y) = exp(\u2212\u221a\nFor any c \u2265 \u2206, we have that\nexp(\u2212c)\nThis suggests that this distribution is especially suited for \u0001-DP.\n\ny). First note that there is no concern about poles, since f (0) is \ufb01nite.\n\nexp(\u2212(cid:112)(c \u2212 \u2206)2)\n\n= exp(\u2206), which is constant.\n\nexp(\u2212(c \u2212 \u2206))\n\nexp(\u2212\n\n\u221a\n\n\u221a\n\nc2)\n\n=\n\nIt is well known in the DP community that Gaussian noise cannot be used to achieve \u0001-DP. We show\nin the next example how Theorem 3 can be used to easily verify this fact.\nExample 5 (Multivariate Normal). The density of a multivariate normal N (\u00b5, \u03a3) has density pro-\nportional to f ((x \u2212 \u00b5)(cid:62)\u03a3\u22121(x \u2212 \u00b5)), where f (y) = exp (\u2212y/2) . If \u2206 > 0, then\n\n(cid:16)\u2212 (c \u2212 \u2206)2 /2\n\n(cid:17)\n\n/ exp(cid:0)\u2212c2/2(cid:1) = lim\n\nc\u2192\u221e exp\n\n(cid:0)c2 \u2212(cid:2)c2 \u2212 2c\u2206 + \u22062(cid:3)(cid:1)(cid:19)\n\n= \u221e.\n\n(cid:18) 1\n\n2\n\nlim\nc\u2192\u221e exp\n\n(cid:20)\n(cid:0)\u2206 +(cid:0)\u221a\n\n2\n\n= 1.\n\n(cid:20)\n\n(cid:21)\n\u22062 + 4\u03bd(cid:1)(cid:1) . Plugging this into\n\n1 + (c \u2212 \u2206)2/\u03bd\n\n1 + c2/\u03bd\n\n=\n\nThe previous result con\ufb01rms that the tails of the Normal distribution are too light to achieve \u0001-DP.\nIn contrast with the previous example, we show next that the multivariate t-distribution can achieve\n\u0001-DP, but its tails are maybe \u201cover-kill\".\nExample 6 (Multivariate t-distribution). A d dimensional t random vector with degrees of freedom\n\u03bd > 1, denoted td\n\n\u03bd(\u00b5, \u03a3) has density proportional to f ((x \u2212 \u00b5)(cid:62)\u03a3\u22121(x \u2212 \u00b5)), where\n(cid:21)(\u03bd+d)/2\n\nf (y) = [1 + y/\u03bd]\n[1 + (c \u2212 \u2206)2/\u03bd]\u2212(\u03bd+d)/2\n\n\u2212(\u03bd+d)/2 .\n\n1 + c2/\u03bd\n\nWe check the limit:\n\nlim\nc\u2192\u221e\n\n[1 + c2/\u03bd]\u2212(\u03bd+d)/2\n\n= lim\nc\u2192\u221e\n\n1 + (c \u2212 \u2206)2/\u03bd\n\nSince the limit is \ufb01nite, we know that there is a \ufb01nite supremum. We solve d\ndc\n0, and \ufb01nd that the unique solution in [\u2206,\u221e) is c = 1\n\n(cid:104)\n\n(cid:105)(\u03bd+d)/2\n\n1+c2/\u03bd\n\ngives us the value of exp(\u0001).\n\n1+(c\u2212\u2206)2/\u03bd\nWe end this subsection with a result for the original in\ufb01nite dimensional problem: if X is in\ufb01nite\ndimensional, then Theorem 3 can be used to achieve \u0001-DP for a set of d linear functionals from K.\nCorollary 1. Assume X is an LCS of potentially in\ufb01nite dimension. Let T : D \u2192 X be a summary\nwith \ufb01nite sensitivity \u2206 < \u221e with respect to an elliptical noise X \u2208 X. Then for any distinct gi \u2208 K\n\u00b5D = {gi(TD)}, \u03a3 = {C(gi, gj)}, and f : [0,\u221e) \u2192 [0,\u221e] is a monotonically decreasing function\ndepending on d and the elliptical family, but not the speci\ufb01c gi. If f (0) < \u221e, and property (2) of\n< \u221e.\n\nfor i = 1, . . . , d, the density of {gi((cid:101)TD)} is proportional to f (\u03c3\u22122(x \u2212 \u00b5D)\u03a3\u22121(x \u2212 \u00b5D)), where\nTheorem 3 holds, then {gi((cid:101)TD)} satis\ufb01es \u0001-DP, where exp(\u0001) = sup\nThe key point of Corollary 1 is that there is a universal \u03c3 such that (cid:101)TD achieves \u0001-DP when evaluated\n\nf ((c \u2212 \u03c3\u22122\u2206)2)\n\non any d linear functionals. Unfortunately, it does depend on d, and as we will see in the next section,\nthere is no \ufb01nite \u03c3 that can guarantee \u0001-DP for arbitrary d when using an elliptical perturbation.\n\nc\u2265\u03c3\u22121\u2206\n\nf (c2)\n\n4.2\n\nImpossibility in In\ufb01nite Dimensional Spaces\n\nIn the previous subsection we gave a condition to check whether an elliptical distribution can be\nused to satisfy \u0001-DP in \ufb01nite dimensional spaces. It is natural to suppose that a similar property\nholds in in\ufb01nite dimensional spaces. However, our main result in this section is that no elliptical\ndistribution satis\ufb01es \u0001-DP in in\ufb01nite dimensional spaces. The intuition behind this result is that by\n\n8\n\n\fCorollary 1, any elliptical process can be expressed as a random mixture of Gaussian processes,\nbut in in\ufb01nite dimensional spaces, the mixing variable V is actually measurable with respect to\nthe in\ufb01nite dimensional process. That is, if one observes \u02dcTD = TD + \u03c3X, then with probability\none, the mixing random variable V can be computed from \u02dcTD. This is because one can pool small\namounts of information across an in\ufb01nite number of dimensions estimate V (even though X still\nisn\u2019t observable). So, the noise from any elliptical distribution is equivalent (as far as privacy goes)\nto adding noise from a Gaussian process, which Mirshani et al. (2017) show only satis\ufb01es (\u0001, \u03b4)-DP,\na weaker notion of differential privacy than \u0001-DP.\n\nTheorem 4. Consider a summary T : D \u2192 X and let (cid:101)TD = TD + \u03c3X, where X is a centered\nsingleton, and C does not have \ufb01nite rank, then (cid:101)TD will not achieve \u0001-DP for any choice of \u03c3.\n(cid:80)n\ni=1 gi((cid:101)TD)2 converges to V 2 with probability 1 as n \u2192 \u221e, recovering V from (cid:101)TD.\n\nelliptical distribution and TD := T (D). If X is in\ufb01nite dimensional, the image T (D) is a not a\n\nProof Sketch. Consider functionals gi \u2208 K such that C(gi, gj) = \u03b4ij. The estimator Vn =\n\n1\nn\n\nFortunately, elliptical distributions can still achieve (\u0001, \u03b4)-DP. However, we run into a bit of an odd\nphilosophical issue since the mixing coef\ufb01cient V can be computed from \u02dcf (D). So, the mechanism\ncan be viewed as drawing from a mixture of Gaussian processes, but after observing the output the\nuser knows exactly from which Gaussian distribution the noise came from.\nTheorem 5. Let X be a centered elliptical process over X and T : D \u2192 X has sensitivity \u2206. Then\n\nfor any \u0001 > 0 and \u03b4 > 0,(cid:101)TD = TD + \u03c3X,\n\nwith\n\n\u03c32 \u2265 2 log(2/\u03b4(cid:48))\n\n\u22062\n\n\u00012\n\nachieves (\u0001, \u03b4)-DP, where \u03b4(cid:48) satis\ufb01es \u03b4 = 2MV (log(\u03b4(cid:48)/2)) and MV is the moment generating func-\ntion of mixing coef\ufb01cient V , as de\ufb01ned in Theorem 1.\nIn Theorem 5, \u03b4(cid:48) represents the DP that would be achieved under the Gaussian mechanism, thus\nIn addition, for \u03b4(cid:48) \u2208 (0, 1), log(\u03b4(cid:48)/2) < 0, so\none will end up with better privacy if \u03b4 < \u03b4(cid:48).\nMV (log(\u03b4(cid:48)/2)) is \ufb01nite and all quantities are well de\ufb01ned. The proof of Theorem 5 is similar to the\nproof of Mirshani et al. (2017, Theorem 3.3), and is in the Supplementary Materials.\n\n5 Discussion\n\nIn this work we considered a new class of additive privacy mechanisms based on elliptical distribu-\ntions. We also presented a number of foundational results concerning the equivalence/orthogonality\nof elliptical distributions. These mechanisms were considered under the general assumption that the\nsummary resides in a locally convex space, allowing for a wide range of applications from classic\nmultivariate statistics to nonparametric statistics and functional data analysis. Surprisingly, we show\nthat while many elliptical distributions may be used for pure DP in \ufb01nite dimensions, none are ca-\npable of achieving it in in\ufb01nite dimensions. This is due to the close connection between Gaussian\nprocesses and elliptical processes, and both can only achieve approximate DP in in\ufb01nite dimensions.\nThis work also highlights the need for more tools when the statistical summaries are complex objects\nsuch as functions. Properties that hold in \ufb01nite dimensions may not hold in in\ufb01nite dimensions in\nsome surprisingly subtle ways. Practically, this can mean that either one does not have the desired\nlevel of protection against privacy disclosures, or that one has to add enormous amounts of noise to\nachieve pure DP.\nWhile this paper has focused on the question of whether an elliptical distribution sats\ufb01es DP, we do\nnot address the utility of these mechanisms. There is already evidence that elliptical distributions\nare useful for different applications (i.e. Awan and Slavkovi\u00b4c, 2019; Bun and Steinke, 2019), but\nfurther work establishing utility guarantees for elliptical distributions is needed.\n\nReferences\nM. Abramowitz and I. A. Stegun. Handbook of mathematical functions: with formulas, graphs, and\n\nmathematical tables, volume 55. Courier Corporation, 1965.\n\n9\n\n\fF. Alda and B. I. Rubinstein. The Bernstein mechanism: Function release under differential privacy.\n\nIn AAAI, pages 1705\u20131711. 2017.\n\nJ. Awan, A. Kenney, M. Reimherr, and A. Slavkovi\u00b4c. Bene\ufb01ts and pitfalls of the exponential mech-\nanism with applications to hilbert spaces and functional pca. arXiv preprint arXiv:1901.10864,\n2019.\n\nJ. Awan and A. Slavkovi\u00b4c. Differentially private uniformly most powerful tests for binomial data.\nIn S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors,\nAdvances in Neural Information Processing Systems 31, pages 4208\u20134218. Curran Associates,\nInc., 2018.\n\nJ. Awan and A. Slavkovi\u00b4c. Structure and sensitivity in differential privacy: Comparing k-norm\n\nmechanisms. ArXiv e-prints, 2019. Under Review.\n\nP. Billingsley. Probability and measure. New York Wiley, 1979.\n\nV. I. Bogachev. Gaussian measures. 62. American Mathematical Soc., 1998.\n\nD. Bosq. Linear Processes in Function Spaces: Theory and Applications. Lecture Notes in Statistics.\n\nSpringer New York, 2000.\n\nM. Bun and T. Steinke. Average-case averages: Private algorithms for smooth sensitivity and mean\n\nestimation. arXiv preprint arXiv:1906.02830, 2019.\n\nK. Chaudhuri and C. Monteleoni. Privacy-preserving logistic regression. In D. Koller, D. Schuur-\nmans, Y. Bengio, and L. Bottou, editors, Advances in Neural Information Processing Systems 21,\npages 289\u2013296. Curran Associates, Inc., 2009.\n\nK. Chaudhuri, C. Monteleoni, and D. Sarwate. Differentially private empirical risk minimization.\n\nIn Journal of Machine Learning Research, volume 12, pages 1069\u20131109. 2011.\n\nR. Couillet, A. Kammoun, and F. Pascal. Second order statistics of robust estimators of scatter.\napplication to glrt detection for elliptical signals. Journal of Multivariate Analysis, 143:249\u2013274,\n2016.\n\nC. Dwork. Differential privacy. In 33rd International Colloquium on Automata, Languages and\nProgramming, part II (ICALP 2006), volume 4052, pages 1\u201312. Springer Verlag, Venice, Italy,\n2006.\n\nC. Dwork, F. McSherry, K. Nissim, and A. Smith. Calibrating Noise to Sensitivity in Private Data\n\nAnalysis, pages 265\u2013284. Springer Berlin Heidelberg, Berlin, Heidelberg, 2006.\n\nC. Dwork and A. Roth. The algorithmic foundations of differential privacy. Found. Trends Theor.\n\nComput. Sci., 9(3–4):211\u2013407, 2014.\n\nK. W. Fang. Symmetric Multivariate and Related Distributions: 0. Chapman and Hall/CRC, 2017.\n\nG. Frahm, M. Junker, and A. Szimayer. Elliptical copulas: applicability and limitations. Statistics\n\n& Probability Letters, 63(3):275\u2013286, 2003.\n\nM. Gaboardi, H.-W. Lim, R. M. Rogers, and S. P. Vadhan. Differentially private chi-squared hy-\npothesis testing: Goodness of \ufb01t and independence testing. In ICML\u201916 Proceedings of the 33rd\nInternational Conference on International Conference on Machine Learning-Volume 48. JMLR,\n2016.\n\nQ. Geng and P. Viswanath. The optimal noise-adding mechanism in differential privacy.\n\nTransactions on Information Theory, 62(2):925\u2013951, 2016.\n\nIEEE\n\nA. Ghosh, T. Roughgarden, and M. Sundararajan. Universally utility-maximizing privacy mecha-\nnisms. In Proceedings of the Forty-\ufb01rst Annual ACM Symposium on Theory of Computing, STOC\n\u201909, pages 351\u2013360. ACM, New York, NY, USA, 2009.\n\nJ. Goes, G. Lerman, and B. Nadler. Robust sparse covariance estimation by thresholding tyler\u2019s\n\nm-estimator. arXiv preprint arXiv:1706.08020, 2017.\n\n10\n\n\fR. Hall, A. Rinaldo, and L. Wasserman. Differential privacy for functions and functional data. The\n\nJournal of Machine Learning Research, 14(1):703\u2013727, 2013.\n\nM. Hardt and K. Talwar. On the geometry of differential privacy.\n\nIn Proceedings of the Forty-\nsecond ACM Symposium on Theory of Computing, STOC \u201910, pages 705\u2013714. ACM, New York,\nNY, USA, 2010.\n\nJ. Hoffmann-J\u00f8rgensen.\n28(2):257\u2013264, 1972.\n\nExistence of conditional probabilities. Mathematica Scandinavica,\n\nO. Kallenberg. Foundations of modern probability. Springer Science & Business Media, 2006.\n\nV. Karwa, D. Kifer, and A. Slavkovi\u00b4c. Private posterior distributions from variational approxima-\ntions. NIPS 2015 Workshop on Learning and Privacy with Incomplete Data and Weak Supervi-\nsion, 2016.\n\nV. Karwa and A. Slavkovi\u00b4c.\n\nInference using noisy degrees: Differentially private \u03b2-model and\n\nsynthetic graphs. The Annals of Statistics, 44(1):87\u2013112, 2016.\n\nD. Kifer and A. Machanavajjhala. Puffer\ufb01sh: A framework for mathematical privacy de\ufb01nitions.\n\nACM Transactions on Database Systems (TODS), 39(1):3, 2014.\n\nD. Kifer, A. Smith, and A. Thakurta. Private convex empirical risk minimization and high-\n\ndimensional regression. Journal of Machine Learning Research, 1:1\u201341, 2012.\n\nF. McSherry and K. Talwar. Mechanism design via differential privacy. In Proceedings of the 48th\nAnnual IEEE Symposium on Foundations of Computer Science, FOCS \u201907, pages 94\u2013103. IEEE\nComputer Society, Washington, DC, USA, 2007.\n\nA. Mirshani, M. Reimherr, and A. Slavkovic. On the existence of densities for functional data and\n\ntheir link to statistical privacy. arXiv preprint arXiv:1711.06660, 2017.\n\nE. Ollila and E. Raninen. Optimal shrinkage covariance matrix estimation under random sampling\n\nfrom elliptical distributions. IEEE Transactions on Signal Processing, 2019.\n\nW. Rudin. Functional analysis, mcgrawhill. Inc, New York, 1991.\n\nR. Schmidt. Credit risk modelling and estimation via elliptical copulae.\n\n267\u2013289. Springer, 2003.\n\nIn Credit Risk, pages\n\nM. Smith, M. \u00c1lvarez, M. Zwiessele, and N. D. Lawrence. Differentially private regression with\ngaussian processes. In A. Storkey and F. Perez-Cruz, editors, Proceedings of the Twenty-First\nInternational Conference on Arti\ufb01cial Intelligence and Statistics, volume 84 of Proceedings of\nMachine Learning Research, pages 1195\u20131203. PMLR, Playa Blanca, Lanzarote, Canary Islands,\n2018.\n\nI. Soloveychik and A. Wiesel. Tyler\u2019s covariance matrix estimator in elliptical models with convex\n\nstructure. IEEE Transactions on Signal Processing, 62(20):5251\u20135259, 2014.\n\nS. Song, K. Chaudhuri, and A. D. Sarwate. Stochastic gradient descent with differentially private\nupdates. In in Proceedings of the Global Conference on Signal and Information Processing. IEEE,\npages 245\u2013248. 2013.\n\nY. Sun, P. Babu, and D. P. Palomar. Robust estimation of structured covariance matrix for heavy-\ntailed elliptical distributions. IEEE Transactions on Signal Processing, 64(14):3576\u20133590, 2016.\n\nD. Vu and A. Slavkovi\u00b4c. Differential privacy for clinical trial data: Preliminary evaluations.\n\nIn\nProceedings of the 2009 IEEE International Conference on Data Mining Workshops, ICDMW\n\u201909, pages 138\u2013143. IEEE Computer Society, Washington, DC, USA, 2009.\n\nY. Wang, D. Kifer, J. Lee, and V. Karwa. Statistical approximating distributions under differential\n\nprivacy. Journal of Privacy and Con\ufb01dentiality, 8(1), 2018.\n\nL. Wasserman and S. Zhou. A statistical framework for differential privacy. Journal of the American\n\nStatistical Association, 105(489):375\u2013389, 2010.\n\n11\n\n\fO. Williams and F. Mcsherry. Probabilistic inference and differential privacy.\n\nIn J. D. Lafferty,\nC. K. I. Williams, J. Shawe-Taylor, R. S. Zemel, and A. Culotta, editors, Advances in Neural\nInformation Processing Systems 23, pages 2451\u20132459. Curran Associates, Inc., 2010.\n\nF. Yu, M. Rybar, C. Uhler, and S. E. Fienberg. Differentially-private logistic regression for detecting\nmultiple-snp association in gwas databases. In Privacy in Statistical Databases: UNESCO Chair\nin Data Privacy, International Conference, PSD 2014, Ibiza, Spain, September 17-19, 2014. Pro-\nceedings, pages 170\u2013184. Springer International Publishing, Cham, 2014.\n\nJ. Zhang, Z. Zhang, X. Xiao, Y. Yang, and M. Winslett. Functional mechanism: Regression analysis\n\nunder differential privacy. Proc. VLDB Endow., 5(11):1364\u20131375, 2012.\n\n12\n\n\f", "award": [], "sourceid": 5380, "authors": [{"given_name": "Matthew", "family_name": "Reimherr", "institution": "Pennsylvania State University"}, {"given_name": "Jordan", "family_name": "Awan", "institution": "Penn State University"}]}