{"title": "Generalization in Generative Adversarial Networks: A Novel Perspective from Privacy Protection", "book": "Advances in Neural Information Processing Systems", "page_first": 307, "page_last": 317, "abstract": "In this paper, we aim to understand the generalization properties of generative adversarial networks (GANs) from a new perspective of privacy protection. Theoretically, we prove that a differentially private learning algorithm used for training the GAN does not overfit to a certain degree, i.e., the generalization gap can be bounded. Moreover, some recent works, such as the Bayesian GAN, can be re-interpreted based on our theoretical insight from privacy protection. Quantitatively, to evaluate the information leakage of well-trained GAN models, we perform various membership attacks on these models. The results show that previous Lipschitz regularization techniques are effective in not only reducing the generalization gap but also alleviating the information leakage of the training dataset.", "full_text": "Generalization in Generative Adversarial Networks:\n\nA Novel Perspective from Privacy Protection\n\nBingzhe Wu1, Shiwan Zhao2, ChaoChao Chen3, Haoyang Xu1\n\nLi Wang3, Xiaolu Zhang3, Guangyu Sun1,4\u2217, Jun Zhou3\n\n1Peking University, 2IBM Research, 3Ant Financial,\n\n4 Advanced Institute of Information Technology, Peking University\n\n{wubingzhe, xuhaoyang, gsun}@pku.edu.cn\n\n{zhaosw}@cn.ibm.com\n\n{chaochao.ccc, aymond.wangl, yueyin.zxl, jun.zhoujun}@ant\ufb01n.com\n\nAbstract\n\nIn this paper, we aim to understand the generalization properties of generative\nadversarial networks (GANs) from a new perspective of privacy protection. Theo-\nretically, we prove that a differentially private learning algorithm used for training\nthe GAN does not over\ufb01t to a certain degree, i.e., the generalization gap can be\nbounded. Moreover, some recent works, such as the Bayesian GAN, can be re-\ninterpreted based on our theoretical insight from privacy protection. Quantitatively,\nto evaluate the information leakage of well-trained GAN models, we perform vari-\nous membership attacks on these models. The results show that previous Lipschitz\nregularization techniques are effective in not only reducing the generalization gap\nbut also alleviating the information leakage of the training dataset.\n\n1\n\nIntroduction\n\nIn the past years, generative adversarial networks (GANs) [12] have achieved remarkable progress in\na wide range of applications including image translation [41, 17], image manipulation [40, 6], and\nimage super-resolution [37, 21], etc. More recently, numerous advanced techniques [1, 22, 23, 39]\nare proposed for improving and stabilizing the training of GANs, leading to more realistic generated\nimages.\nDespite the tremendous success, there are still numerous open problems to be fully solved, ranging\nfrom the theoretical analysis of different regularization techniques [23, 29] to the visualization of\ndifferent objective functions of GANs [3]. Among these problems, a critical one is how to formally\ncharacterize the generalization ability of GANs. Some recent studies attempted to explore this\nproblem in different contexts. For instance, a seminal work in this direction [2] proposed the neural\nnet distance, and the authors further showed the generalization properties of this distance. Qi et al.\n[26] were motivated by the progress in the Lipschitz regularization and proposed a loss-sensitive\nGAN. They then developed the Lipschitz regularization theory to analyze the generalization ability\nof the loss-sensitive GAN.\nDifferent from the prior works [26, 2], in this paper, we aim to study the generalization ability of\nGANs in a relatively general setting from a novel perspective of privacy protection. Our study is\nmotivated by a well-known intuition [38], \u201creducing the generalization gap\u201d and \u201cprotecting the\nindividual\u2019s privacy\u201d share the same goal of encouraging a neural network to learn the population\u2019s\nfeatures instead of memorizing the features of each individual, i.e., the smaller the generalization gap\n\u2217Corresponding author. This work is supported by NSF China 61832020, NSF China 61572045, and Beijing\n\nAcademy of Arti\ufb01cial Intelligence.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fis, the less information of the training dataset will be revealed. The goal of this paper is to validate\nthis natural intuition theoretically and quantitatively.\nIn the theoretical side, we leverage the stability-based theory [31] to bridge the gap between differen-\ntial privacy [11] and the generalization, i.e., a differentially private learning algorithm does not over\ufb01t\nto a certain degree. Based on the theoretical analysis, we also provide a new perspective from privacy\nprotection to understand a number of recent techniques for improving the performance of GANs, e.g.,\nvarious Lipschitz regularization terms [13, 23] and training GANs using Bayesian learning [28, 15].\nIn the experimental side, we quantitatively validate the relationship between the generalization gap\nand the information leakage of the training dataset. To this end, we introduce the membership attack\n[32] to evaluate the information leakage of a trained GAN model. In the context of machine learning,\nthe membership attack refers to inferring whether a speci\ufb01c item is from the training dataset while\ngiven the trained model (discriminator and generator in our case). Speci\ufb01cally, we design different\nattack models and perform membership attacks on GANs trained with various objective functions and\nregularization terms. The results show that previous Lipschitz regularization techniques are effective\nin not only reducing the generalization gap but also alleviating the information leakage of the training\ndataset, which implicitly validates the aforementioned intuition. The results also suggest that it is\npossible to design new variants of GAN from the perspective of building privacy-preserving learning\nalgorithms, which can bring signi\ufb01cant regularization effects while provide appealing property of\nprotecting the sensitive information contained in the training dataset.\nThe rest of this paper is organized as follows. We \ufb01rst brie\ufb02y review related works in Section 2.\nThen we demonstrate our theoretical analysis in Section 3. Subsequently, we present the quantitative\nanalysis in Section 4. At last, we conclude this work in Section 5.\n\n2 Related Work\n\nGenerative adversarial networks.\nIn the past years, starting from the illuminative work of\nGANs [12], many efforts have been devoted to this research area. Numerous researchers have\ntried to improve the performance of GANs from different perspectives [1, 26, 25, 23]. One direction\nis to improve the original objective function [26, 1, 25]. For example, to solve the gradient vanishing\nproblem, the Least-square GAN [22] proposed using the least square loss function in the training of\nGANs. Wasserstein GAN (WGAN) [1] replaced the original Jensen\u2013Shannon (JS) divergence with\nthe Wasserstein distance and proposed an effective approximation method to compute the distance.\nBesides the improvements on the objective function, lots of algorithmic tricks have been proposed\nin empirical studies of training GANs. A typical direction is to add Lipschitz constraints on the\ndiscriminator, which enables the discriminator to be Lipschitz continuous with respect to the input\nand the weight. For instance, WGAN proposed using weight clipping to constrain the Lipschitz\nconstant [1]. Gulrajani et al. [13] further suggested to use the gradient penalty to obtain better\nperformance. Miyato et al. [23] took a different angle to regularize the spectra of the weight matrix\nwhich can implicitly constrain the Lipschitz constant.\nAmong these empirical techniques in training GANs, some researchers focus on building the theoreti-\ncal framework to analyze the properties of GANs under different assumptions. Here we focus on the\nworks of studying the generalization properties of GANs. Speci\ufb01cally, Arora et al. [2] argued that the\nobjective function of the original GAN does not provide a theoretical generalization guarantee. As a\nresult, the authors turned into analyzing the generalization gap of their proposed neural network dis-\ntance. Qi et al. [26] developed a set of theoretical tools to analyze the generalization of GANs under\nthe assumption of Lipschitz continuous of the discriminator. In practical, to meet the assumption,\nthey designed a novel objective function to directly minimize the Lipschitz constant.\nMembership attacks towards deep learning algorithms. Recently, membership attacks have\narisen as a common threat model against machine learning algorithms and attained increasing\nattraction from the research community [33, 32, 5, 14]. A pioneer work [32] investigated the risk of\nmembership attacks on different machine learning models. Speci\ufb01cally, they developed a shadow\ntraining technique to obtain an attack model in the black-box setting (i.e., without knowing the\nmachine learning model structure and parameters). Carlini et al. [5] proposed a metric to measure the\nvulnerability of deep learning models. In the context of GANs, Hayes et al. [14] studied membership\nattacks against GANs in both black-box and white-box settings. In this paper, we use the membership\nattack to assess the information leakage of the dataset used for training GAN models.\n\n2\n\n\f3 Theoretical Analysis\n\nIn this part, we aim to bridge the gap between the privacy protection and the generalization ability\nof GANs. At a high level, we prove that a differentially private learning mechanism (i.e. training\nalgorithm) does not over\ufb01t to a certain extent. The core idea is based on the stability-based theory [31].\nTo this end, we \ufb01rst introduce some basic notations in GANs. Then we present the theoretical analysis\nto characterize the generalization ability of GANs from the perspective of privacy protection. At last,\nbased on our theoretical insight from privacy protection, we show a new interpretation of the previous\ntheoretical results of uniform convergence [2], as well as the recent efforts on the Bayesian GAN.\nPreliminaries. The GAN framework consists of a generator and a discriminator. We denote HG and\nHD as the hypothesis spaces of the generator and discriminator, respectively. In practice, we make\nuse of neural networks to build HG and HD. Formally, we have HG = {g(z; \u03b8g), \u03b8g \u2208 Rp} and\nHD = {d(x; \u03b8d), \u03b8d \u2208 Rq}, where g and d are multi-layer convolutional neural networks. \u03b8g and \u03b8d\nare the corresponding weight parameters. The training of GANs can be seen as playing a min-max\ngame to solve the following optimization problem:\n\nmin\n\u03b8g\n\nmax\n\n\u03b8d\n\nEx\u223cpdata [\u03c6(d(x; \u03b8d))] + Ez\u223cpz [\u03c6(1 \u2212 d(g(z; \u03b8g); \u03b8d))]\n\n(1)\n\nThe above formulation can be seen as an extended version of the objective function used in the vanilla\nGAN [12]. According to previous literature [2], we call function \u03c6 the measuring function. Note that\nsetting \u03c6(t) = log(t) leads to the objective function used in the work of the original GAN [12], while\nthe recent WGAN [1] proposed using \u03c6(t) = t.\nTo optimize Equation 1, we need to build two learning mechanisms Ad and Ag. During the training,\nwe alternately run Ad and Ag to seek an equilibrium (\u03b8\u2217\nd such\nthat it maximizes the expected loss of the discriminator:\n\ng). Speci\ufb01cally, Ad tries to \ufb01nd \u03b8\u2217\n\nd, \u03b8\u2217\n\nU (\u03b8d, \u03b8\u2217\nand Ag tries to \ufb01nd \u03b8\u2217\n\ng) = Ex\u223cpdata [\u03c6(d(x; \u03b8d))] + Ez\u223cpz [\u03c6(1 \u2212 d(g(z; \u03b8\u2217\ng to minimize the expected loss of the generator:\nd, \u03b8g) = Ez\u223cpz [\u03c6(1 \u2212 d(g(z; \u03b8g); \u03b8\u2217\n\nV (\u03b8\u2217\n\nd))]\n\ng); \u03b8d))]\n\n(2)\n\n(3)\n\nEmpirical loss and generalization. The optimization of Equation 2 and 3 can not be directly solved\nsince the expectation over the distribution of the true data pdata is intractable. Instead, we approximate\nthem with empirical loss on a set of i.i.d. real data samples S = {x1, x2,\u00b7\u00b7\u00b7 , xm} and noise vectors\nZ = {z1, z2,\u00b7\u00b7\u00b7 , zm} drawn from pdata and pz, respectively2. We denote the resulted empirical\nversions of Equation 2 and 3 as \u02c6U and \u02c6V . In this empirical setting, the learning mechanisms Ad and\nAg turn into the role as the empirical loss optimizers, which are to optimize the empirical loss \u02c6U and\n\u02c6V , i.e., \ufb01nding (\u02c6\u03b8\u2217\ng). To study the generalization property of the learning mechanisms, we need to\nevaluate the generalization gap between the empirical and expected objective losses. In this paper, we\nmainly focus on the analysis of Equation 2 since our viewpoint is from the privacy protection and\nEquation 3 does not explicitly touch the original training data. As a common practice shown in the\nprior work [12], we analyze Equation 2 when \u03b8\u2217\ng is given. Formally, we de\ufb01ne the generalization gap\nas follows (we take the discriminator loss U as an example):\n\nd, \u02c6\u03b8\u2217\n\n[ \u02c6U (\u03b8d, \u03b8\u2217\n\ng) \u2212 U (\u03b8d, \u03b8\u2217\ng)]\n\ndata\n\nFU (Ad) = E\u03b8d\u223cAd(S)ES\u223cpm\n\n(4)\nwhere S \u223c pm\ndata denotes sampling m training samples from the oracle distribution pdata. In the\nabove equation, we take the expectation with respect to the randomness in the learning mechanism\nand also in the sampling process similar to the literature [24, 36]. Note that we can in\ufb01nitely sample\nfrom pz (e.g. a uniform distribution) and pz is irrelevant to the original training data (i.e. sampling\nfrom pz does not induce the leakage of the training dataset). As a result, we omit the second term in\nthe RHS of Equation 2 and focus on studying the \ufb01rst term.\nPrivacy protection and generalization bound. To bridge the gap between privacy protection and\nthe generalization bound, we need to characterize how an algorithm can protect privacy, i.e., the\namount of information leakage of the training dataset. Differential privacy [10] is seen as a gold\nstandard for privacy protection in the security community. It provides a rigorous bound on privacy\n\n2In practice, we always sample different Z while S is \ufb01xed at each training round.\n\n3\n\n\fcost of the algorithm, even in the worst case. The de\ufb01nition of differential privacy is based on the\nadjacent datasets. Two datasets are adjacent when they differ in a single element. Then we can\nintroduce differential privacy as follows:\nDe\ufb01ne 1 (Differential privacy) A randomized algorithm A : D \u2192 R satis\ufb01es \u0001-differential privacy\nif for any two adjacent datasets S,S(cid:48) \u2286 D and for any subset of outputs O \u2286 R it holds:\n\nPr[A(S) \u2208 O] \u2264 e\u0001Pr[A(S(cid:48)\n\n) \u2208 O]\n\n(5)\nIn our setting, A can be the training algorithm (i.e. Ad). Intuitively, Equation 5 indicates that\nparticipation of one individual sample in the training phase has a negligible effect on the \ufb01nal weight\nparameters. A relevant concept is uniform RO-stable of an algorithm. An algorithm is stable if a\nsmall change to the input causes a limited change in the output. Here, RO denotes \u201creplace one\nelement in the input\u201c. The above description is made formally as:\nDe\ufb01ne 2 (Uniform RO-stability) The randomized algorithm A is uniform RO-stable with respect to\nthe discriminator loss function (Equation 2) in our case, if for all adjacent datasets S, S(cid:48), it holds\nthat:\n\n|E\u03b8d\u223cA(S)[\u03c6(d(x; \u03b8d))] \u2212 E\u03b8d\u223cA(S(cid:48))[\u03c6(d(x; \u03b8d))]| \u2264 \u0001stable(m)\n\nsup\nx\u2208S\n\n(6)\n\nA well-known heuristic observation is that differential privacy implies uniform stability. The prior\nwork [36] has formlized this observation into the following lemma:\nLemma 1 (Differential privacy \u21d2 uniform RO-stability) If a randomized algorithm A is \u0001-\ndifferentially private, then the algorithm A satis\ufb01es (e\u0001 \u2212 1)-RO-stability.\nThe stability of the algorithm is also related to the generalization gap. Numerous studies [31, 24]\nfocus on exploring the relationship in various settings. Formally, we have the following lemma:\nLemma 2 If an algorithm A is uniform RO-stable with rate \u0001stable(m), then |FU (A)| (Equation 4)\ncan be bounded: |FU (A)| \u2264 \u0001stable(m).\nIntuitively, the more stable the algorithm is, the better its generalization ability will be. We take a\nfurther step to build the connection between differential privacy and the generalization gap. This can\nbe done via combining the above two lemmas. Formally, we introduce Theorem 1 as follows:\nTheorem 1 (Generalization gap) If an algorithm A satis\ufb01es \u0001-differential privacy, then the general-\nization gap can be bounded by a data-independent constant.\nThe proof can be accomplished by following the roadmap: Dif f erential privacy \u21d2 Stability \u21d2\nGeneralization. The proof details can be found in Appendix. By applying Theorem 1 to Ad,\nwe can show that the generalizability of the discriminator is ensured when the training algorithm\nsatis\ufb01es \u0001-differential privacy. Note that we focus on the generalization of the discriminator loss,\nsince the optimization of Equation 3 does not touch the original data. We can derive the similar\ngeneralization bound of the generator by leveraging the post-processing property of the differential\nprivacy protocol [11], with the help of the robust generalization notations in the adaptive learning\nparadigm [7].\nTheorem 1 not only enables characterizing GAN\u2019s generalizability from the viewpoint of privacy\nprotection, but also helps to understand previous techniques for improving the generalization of\nGANs. A typical example is the Lipschitz regularization technique. Previous studies propose\nimplementing the regularization from various angles [1, 26, 23]. For instance, the loss-sensitive\nGAN [26] designed a novel objective loss to restrict the discriminator to satisfy the Lipschitz condition.\nSpectral normalization [23] explored this direction by adding regularization on the weight parameters.\nAnd WGAN [1] proposed using gradient penalty to constrain the magnitude of the gradient, which\nfurther implicitly led to the Lipschitz condition. From the perspective of differential privacy, the\nLipschitz condition for the outputs is also a crucial ingredient for building a differentially private\nalgorithm. We also infer that adding Lipschitz constraints implicitly leads to stability of the algorithm,\nwhich can further be used for reducing the generalization gap (see more details in the evaluation\nsection).\n\n4\n\n\fAbove analysis focuses on the discriminator loss U. As mentioned above, it is natural to extend the\nanalysis to the generator loss since the optimization of the generator loss does not touch the original\ntraining dataset and we can leverage the post-processing property of the differential privacy. We can\nfurther study the whole optimization procedure of the GAN, i.e., the alternative running of Ad and\nAg. These can be accomplished by the composition theory in adaptive learning theory [8], we mark\nthis as the future work.\nRevisiting previous results on uniform convergence. Previous works attempted to explore the\nuniform convergence with respect to different objective functions. For example, Qi et al. [26] proposed\nthe loss-sensitive GAN and proved the uniform convergence with respect to the discriminator loss.\nArora et al. proposed the neural distance to analyze the generalization property (uniform convergence)\nof GANs. Note that both of them focus on the special form of the GAN or the objective function. In\nthis paper, based on the aforementioned theoretical results, we can prove the uniform convergence of\nthe discriminator loss when the training algorithm satis\ufb01es the differential privacy protocol. Formally,\nwe have the following theorem:\nTheorem 2 (Uuniversal Bound ): Suppose Ad satis\ufb01es \u0001-differential privacy and d(k)(x; \u03b8(k)\nd ) be\nthe output of Ad at the k-th iteration. Then, \u2200k, the generalization gap with respect to dk can be\nbounded by a universal constant which is related to \u0001.\n\nThe proof of the above theorem can be done via combing the post-processing property [11] of\ndifferential privacy and McDiarmid\u2019s inequality [34]. The details can be found in Appendix.\nConnection to Bayesian GAN. Recently, training GANs using Bayesian learning has emerged as\na new way to avoid mode collapse [28]. A well-known interpretation of mode collapse is that\nthe generator/discriminator has memorized some examples from the training dataset. Hence, the\nmemorization phenomenon can also breach the privacy of individuals in the training dataset. Thus,\nwe infer that the effectiveness of the Bayesian GAN may come from preventing information leakage\nof the training dataset. In what follows, we brie\ufb02y introduce how we validate this conjecture with\nour theoretical results. Speci\ufb01cally, we take a recent work [28] as an example. In the work, the\nauthors proposed using stochastic Hamiltonian Monte Carlo (HMC) to marginalize the posterior\ndistribution of the weight parameters of both generator and discriminator. We have noted that Wang\net al. [35] pointed out that sampling one element from a posterior distribution can implicitly satisfy\ndifferential privacy. Based on their results, we can prove that the HMC sampling also preserves\ndifferential privacy with minor modi\ufb01cation (refers to Section 4 in [35]), and then the Bayesian GAN\ncan implicitly preserve the differential privacy. Thus we are not surprised that the Bayesian GAN can\nalleviate mode collapse in GANs since its connection to differential privacy mentioned above.\n\n4 Quantitative Analysis\n\nIn this section, we quantitatively validate the connection between the generalizability and the privacy\nprotection/information leakage of GANs by investigating some of the most popular GAN models. In\nthe theoretical analysis, we focus on the differentially private learning algorithms. While in practice,\ndifferential privacy is a strict requirement for most of the existing GANs so that we focus on studying\ninformation leakage of GANs instead. In particular, we note that adding Lipschitz constraints on the\ndiscriminator has recently emerged as an effective solution for improving the generalization ability\nof such GANs, thus we aim to study the effects of various regularization techniques for adding the\nLipschitz constraints. In a nutshell, our results show that the Lipschitz constraints not only reduce the\ngeneralization gap but also make the trained model resistant to the membership attacks which are\nused for detecting the sensitive information contained in the training dataset.\nThis section is structured as follows. First, we will introduce the experimental settings including the\ndatasets and the choice of different hyper-parameters. Then, we demonstrate our main results on\ndifferent datasets. At last, we provide some discussions on the attack methods and the regularization\ntechniques.\n\n4.1 Experimental Setup\nDatasets. We conduct experiments on a face image dataset and a real clinical dataset, namely,\nLabeled Faces in the Wild (LFW) [20] which consists of 13,233 face images, and the IDC dataset\n\n5\n\n\fwhich is publicly available for invasive ductal carcinoma (IDC) classi\ufb01cation3 and contains 277,524\npatches of 50 \u00d7 50 pixels (198,738 IDC-negative and 78,786 IDC-positive).\nModel setup. Note that we focus on studying the effects of different regularization techniques\ninstead of the architecture design of the GAN model, thus we use the same generator architecture and\nthe same discriminator architecture in all experiments. Speci\ufb01cally, we adopt DCGAN following\nmost of previous works [27]. More details can be found in the work [27]. We set the size of the\ngenerated image to 64x64 for the LFW dataset and 32x32 for the IDC dataset, respectively. As for\noptimization, we use Adam [18] in all experiments, and use different hyper-parameters for different\ntraining strategies. To be speci\ufb01c, we make use of Adam for the GAN trained with JS divergence.\nThe learning rates is set to 0.0004 for the GAN trained without any regularization terms (original\nGAN [12]), while for other GANs (e.g. trained using Wasserstein distance), the learning rate is set to\n0.0002. More details of hyper-parameter settings (e.g. \u03b2 in Adam) can be found in Appendix. We\ntrained all the models for 400 epochs on both datasets.\nAttack setup. We make use of membership attacks for evaluating the information leakage of the\nGAN model. We build the attack model based on the output of the discriminator, which is a bounded\nfunction. We suppose d(x; \u03b8d) \u2264 b for all x in Equation 1. This assumption naturally holds in the\ncontext of GAN (e.g. b = 1 for the original GAN). This is also a common assumption in many\nprevious works. Here, letting b = 1 suf\ufb01ces to all our cases. We then assume that the attacker A\nhas access to the trained GAN model, i.e. the discriminator and generator. Note that the notation\nA is different from the previous ones that denote training algorithms. The goal of the attacker is to\ndetermine whether a record (an image in our case) in the attack testing dataset is from the original\ntraining dataset. Based on the above setting, the attack model proceeds as follows:\n\n\u2022 Given the discriminator d(x; \u03b8d) and an image from the attack testing dataset.\n\u2022 A \ufb01rstly sets a threshold t \u2208 (0, 1).\n\u2022 A outputs 1 if d(x; \u03b8d)/b \u2265 t, otherwise, it outputs 0.\n\nwhere the output of 1 indicates that the input image is from the training dataset.\nTo evaluate the performance of the attack model, we need to build the attack testing dataset. For the\nLFW dataset, we randomly choose 50% of images as the original training dataset to train the GAN\nmodel. We build the attack testing dataset by mixing the original training dataset and the remaining\nimages. For the IDC dataset, we only use the positive part of the dataset. Since the data provider\nhas already provided a partition of training and testing datasets [16] (22,383 images in the training\ndataset and 14,157 images in the testing dataset4), we directly use the original partition and build the\nattack testing dataset by mixing the training and testing datasets. We then treat the above attack as a\nbinary classi\ufb01cation model and evaluate its performance based on the F1 score and the AUC value.\nTo compute the F1 score, we assume the attacker has obtained the average value of d(x; \u03b8d) on the\ntraining dataset. Thus we can set the average value as the threshold t and then compute the F1 score\nat this threshold.\n\n4.2 Results on LFW and IDC Datasets\n\nHere, we present the results of our evaluation on the GAN models trained with different strategies. We\nconduct extensive experiments of different settings. Speci\ufb01cally, we focus on three commonly used\ntechniques for adding Lipschitz constraints, namely, weight clipping [1], gradient penalty [29], and\nspectral normalization [23]. For weight clipping, we set the clip interval as [\u22120.01, 0.01] following\nthe prior work [1]. We combine these regularization techniques with two types of objective functions,\nthe traditional JS divergence (setting \u03c6(t) = log(t)) and the Wasserstein distance (setting \u03c6(t) = t).\nAs mentioned above, the performance of the attack model is measured by the F1 score and AUC\nvalue, and we use the gap between the testing and training losses to estimate the generalization gap.\nWe also calculate the Inception score [29], which is a commonly used for assessing the quality of the\ngenerated images in previous works. The overall results are shown in Table 1.\nLFW. We \ufb01rst conduct contrastive experiments on the GAN trained using JS divergence (setting\n\u03c6(x) = log(x) in Equation 1). As shown in Table 1, the plain model (trained without any regulariza-\n\n3http://www.andrewjanowczyk.com/use-case-6-invasive-ductal-carcinoma-idc-segmentation/\n4They also provide a validation dataset and we did not use it in our experiments.\n\n6\n\n\fTable 1: Evaluation results of DCGAN trained with different strategies. IS denotes the Inception\nscore. N/A indicates that the strategy leads to failure/collapse of the training. The last row presents\nthe Inception scores of the real data (training images of these two datasets).\n\nStrategy\n\n-JS divergence-\n\nOriginal\n\nWeight Clipping\n\nSpectral Normalization\n\nGradient Penalty\n-Wasserstein-\nW/o clipping\n\nWeight Clipping\n\nSpectral Normalization\n\nGradient Penalty\nIS (Real data)\n\nF1\n\n0.565\n0.486\n0.482\n\n0.484\n0.515\n0.492\n\nLFW\n\nAUC\n\nGap\n\nIS\n\nF1\n\nIDC\n\nAUC\n\nGap\n\nIS\n\n0.729\n0.501\n0.506\n\n0.581\n0.113\n0.106\n\nN/A\n\nN/A\n\n0.512\n0.505\n0.503\n\n0.042\n0.017\n0.031\n\n3.067\n3.112\n3.104\n\n0.445\n0.378\n0.416\n\n3.013\n3.156\n2.994\n\n0.388\n0.415\n0.426\n\n0.531\n0.502\n0.508\n\n0.138\n0.053\n0.124\n\nN/A\n\nN/A\n\n0.513\n0.507\n0.504\n\n0.045\n0.013\n0.017\n\n4.272\n\n3.061\n\n2.148\n2.083\n2.207\n\n1.912\n2.196\n1.974\n\ntion term) is more vulnerable than those trained with different regularizers. The plain model leaks\nsome information of the training dataset, which results in the F1 score of 0.565 and the AUC value of\n0.729 (greater than 0.5), respectively. While various regularization techniques are used for training\nthe GAN, the attack performance decreases drastically. For instance, while spectral normalization is\nused, the F1 score is dropped from 0.565 to 0.482, and the AUC value is dropped to 0.506, which\napproximates to the random guess. Along with the decrease of the attack performance, we also\nobserve a consistent decrease in the generalization gap (Gap in Table 1). For example, the gap is\ndecreased from 0.581 to 0.106 while spectral normalization is used.\nIn addition to these metrics, we also calculate the Inception score of each model. As shown in Table 1,\nthe Inception score of the LFW dataset (shown in the last row of Table 1) is much smaller than the\ncommonly used benchmarks (e.g. Cifar10 [19] and ImageNet [9]). This is because that the Inception\nscore is calculated by the model pre-trained by the images in ImageNet in which the image always\ncontains general objects (such as animals) while the image in LFW always contains one face. We\nobserve that the spectral normalization can generate images with higher visual qualities but obtain\nlower Inception score than the weight clipping strategy (see generated images in Appendix). Thus,\nthese numerical results of Inception score indicate that Inception score is not suitable for some image\ngeneration tasks and we need to design a speci\ufb01c metric for a given task. Among these experiments,\nwe also conduct the same attack experiments on GANs trained using Wasserstein distance and observe\nsimilar phenomena as shown in Table 1.\nIDC. The images in the IDC dataset contain various tissues and all of these tissues have similar\nshapes and colors. As a result, performing the attack on the IDC dataset is more challenging than\non other datasets which consist of some concrete objects (e.g. LFW and Cifar10 [19]). As we can\nsee in Table 1, in all cases, the attack performance on IDC is lower than the performance on LFW in\nterms of the F1 score and AUC value. We also provide some quantitative analysis to interpret the\nperformance drop in the following subsection. Despite the performance drop, we can still observe the\neffectiveness of different regularizers in reducing the generalization gap and information leakage.\nFor example, with the use of spectral normalization, AUC drops from 0.531 to 0.508 and F1 score\ndecreases from 0.445 to 0.416. Another notable thing is that training the GAN using Wasserstein\ndistance without weight clipping can lead to the failure of training. This may be caused by the\ngradient explosion since there is no activation function to compress the output values of the last layer\nof the discriminator in Wasserstein GAN (in contrast, the original GAN used the sigmoid function for\nthe compression purpose). For the results of Inception score, we can observe an obvious decrease\nfrom the LFW dataset to the IDC dataset. This may be caused by the difference of contents contained\nin the images (tissues in IDC images and faces in LFW images).\nIn summary, all the empirical evidence implies that previous regularizers for adding Lipschitz\nconstraints can not only decrease the generalization gap but also reduce the information leakage of the\ntraining dataset in terms of the attack performance. Moreover, we observe that spectral normalization\n\n7\n\n\fachieves comparable visual quality of the generated images in contrast to the original GAN (see\ngenerated images in Appendix). This suggests that we can attempt to use this technique in various\napplications to trade off the information leakage and the visual quality of the generated images.\n\n4.3 Discussions\nAttack performance on different datasets. In the experiments, we observe that the attack perfor-\nmance may vary on different datasets. From Table 1, in all cases, the attack performance of IDC is\nalways worse than the performance of LFW. For the original GAN model, the AUC of the attack\nmodel on LFW is 0.729 while the one for IDC is 0.531 which shows a 27.2% relative performance\ndrop. We infer that the drop is caused by the higher similarity of the images in the IDC data. Quanti-\ntatively, the similarity can be measured by the standard deviations (per channel for an RGB image) of\nthese two datasets. Speci\ufb01cally, the standard deviation of IDC is 0.085 (R-channel) while the value of\nLFW is 0.249 (R-channel). The IDC dataset has signi\ufb01cantly lower standard deviation than the LFW\ndataset. This suggests that an individual example in the IDC dataset will be less likely to noticeably\nimpact the \ufb01nal model which further constrains an adversary\u2019s ability. Note that similar evidence has\nbeen found in the prior work [32].\nOther weight normalization. In the previous experiments, we have employed three regulariz-\ners for limiting the discriminator to be Lipschitz continuous.\nIn addition to these regularizers,\nthere are some other approaches to use weight normalization techniques to regularize the model\ntraining. The original weight normalization introduced in the work [30] is to normalize the l2\nnorm of each row vector in a weight matrix. Subsequently, Brock et al. [4] proposed orthonormal\nregularization on the weight matrix to stabilize the training of GANs. We also conduct attack\nexperiments on the GANs trained with these two normalization methods. In practice, the orthonor-\nmal regularization achieves comparable performance in terms of the F1 score of the attack model\n(0.402 for the IDC dataset), while obtains comparable image quality compared with spectral nor-\nmalization. In addition, the original weight normalization will lead to training failure (not conver-\ngence/mode collapse) in our cases. The failure may be caused by the con\ufb02ict between the weight\nnormalizing and the desire to use as many features as possible as discussed in the prior work [23].\n\nStrategy\nOriginal\n\nAUC\n0.549\n0.502\n0.497\n\nGap\n0.581\n0.113\n0.106\n\ni.e.,\n\nWeight Clipping\n\nSpectral Normalization\n\nF1\n0.423\n0.358\n0.347\n\nTable 2: The results of black-box attack on the LFW\ndataset.\n\nBlack-box attack. The aforementioned\nanalysis is all based on the white-box\nattack,\nthe model structure and\nweights of the discriminator and genera-\ntor are exposed to the adversary. In this\npart, we present some discussions on the\nblack-box attack. We consider black-box\nattacks where the adversary has limited\nauxiliary knowledge of the dataset fol-\nlow the prior work [14]. Speci\ufb01cally, for the LFW dataset, we assume the attacker has 30% images\nof both the training and testing datasets (marked as the auxiliary knowledge in the following part).\nThen we can use the auxiliary information to build a \"fake\" discriminator to imitate the behavior\nof the original discriminator (more details can be found in the prior work [14]). Once the \"fake\"\ndiscriminator is obtained, we can replace the original discriminator with the fake one to perform the\naforementioned white-box attack. The results of the black-box attack are summarized in Table 2.\nIntuitively, the black-box is a more challenging task than the white-box attack. This can be observed\nby the decrease of the attack performance from the white-box to black-box (the AUC decreases from\n0.729 to 0.549 for the original model). From Table 2, we can also observe the effectiveness of the\nspectral normalization for reducing the information leakage.\n5 Conclusion\nIn this paper, we have shown a new perspective of privacy protection to understand the generalization\nproperties of GANs. Speci\ufb01cally, we have validated the relationship between the generalization\ngap and the information leakage of the training dataset both theoretically and quantitatively. This\nnew perspective can also help us to understand the effectiveness of previous techniques on Lipschitz\nregularizations and the Bayesian GAN. We hope our work can light the following researchers to\nleverage our new perspective to design GANs with better generalization ability while preventing the\ninformation leakage.\n\n8\n\n\fReferences\n[1] Martin Arjovsky, Soumith Chintala, and L\u00e9on Bottou. Wasserstein gan. arXiv preprint\n\narXiv:1701.07875, 2017.\n\n[2] Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. Generalization and\nequilibrium in generative adversarial nets (gans). In Proceedings of the 34th International\nConference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017,\npages 224\u2013232, 2017.\n\n[3] David Bau, Jun-Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B. Tenenbaum, William T.\nFreeman, and Antonio Torralba. Visualizing and understanding generative adversarial networks.\nIn International Conference on Learning Representations, 2019.\n\n[4] Andrew Brock, Theodore Lim, James M. Ritchie, and Nick Weston. Neural photo editing with\nintrospective adversarial networks. In 5th International Conference on Learning Representations,\nICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017.\n\n[5] Nicholas Carlini, Chang Liu, Jernej Kos, \u00dalfar Erlingsson, and Dawn Song. The secret sharer:\nMeasuring unintended neural network memorization and extracting secrets. ArXiv e-prints,\n1802.08232, 2018.\n\n[6] Yang Chen, Yu-Kun Lai, and Yong-Jin Liu. Cartoongan: Generative adversarial networks for\nphoto cartoonization. In 2018 IEEE Conference on Computer Vision and Pattern Recognition,\nCVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 9465\u20139474, 2018.\n\n[7] Rachel Cummings, Katrina Ligett, Kobbi Nissim, Aaron Roth, and Zhiwei Steven Wu. Adaptive\nlearning with robust generalization guarantees. In Proceedings of the 29th Conference on\nLearning Theory, COLT 2016, New York, USA, June 23-26, 2016, pages 772\u2013814, 2016.\n\n[8] Rachel Cummings, Katrina Ligett, Kobbi Nissim, Aaron Roth, and Zhiwei Steven Wu. Adaptive\nlearning with robust generalization guarantees. In Proceedings of the 29th Conference on\nLearning Theory, COLT 2016, New York, USA, June 23-26, 2016, pages 772\u2013814, 2016.\n\n[9] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Fei-Fei Li. Imagenet: A large-scale\nhierarchical image database. In 2009 IEEE Computer Society Conference on Computer Vision\nand Pattern Recognition (CVPR 2009), 20-25 June 2009, Miami, Florida, USA, pages 248\u2013255,\n2009.\n\n[10] Cynthia Dwork. Differential privacy. Encyclopedia of Cryptography and Security, pages\n\n338\u2013340, 2011.\n\n[11] Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Founda-\n\ntions and Trends in Theoretical Computer Science, 9(3-4):211\u2013407, 2014.\n\n[12] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil\nOzair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani,\nM. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural\nInformation Processing Systems 27, pages 2672\u20132680. Curran Associates, Inc., 2014.\n\n[13] Ishaan Gulrajani, Faruk Ahmed, Mart\u00edn Arjovsky, Vincent Dumoulin, and Aaron C. Courville.\nImproved training of wasserstein gans. In Advances in Neural Information Processing Systems\n30: Annual Conference on Neural Information Processing Systems 2017, 4-9 December 2017,\nLong Beach, CA, USA, pages 5769\u20135779, 2017.\n\n[14] Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. LOGAN: evalu-\nating privacy leakage of generative models using generative adversarial networks. CoRR,\nabs/1705.07663, 2017.\n\n[15] Hao He, Hao Wang, Guang-He Lee, and Yonglong Tian. Bayesian modelling and monte carlo\n\ninference for GAN. In International Conference on Learning Representations, 2019.\n\n[16] Andrew Janowczyk and Anant Madabhushi. Deep learning for digital pathology image analysis:\n\nA comprehensive tutorial with selected use cases. Journal of pathology informatics, 7, 2016.\n\n9\n\n\f[17] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative\n\nadversarial networks. CoRR, abs/1812.04948, 2018.\n\n[18] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In 3rd\nInternational Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May\n7-9, 2015, Conference Track Proceedings, 2015.\n\n[19] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images.\n\nTechnical report, Citeseer, 2009.\n\n[20] Gary B. Huang Erik Learned-Miller. Labeled faces in the wild: Updates and new reporting\nprocedures. Technical Report UM-CS-2014-003, University of Massachusetts, Amherst, May\n2014.\n\n[21] Christian Ledig, Lucas Theis, Ferenc Huszar, Jose Caballero, Andrew Cunningham, Alejandro\nAcosta, Andrew P. Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, and Wenzhe Shi.\nPhoto-realistic single image super-resolution using a generative adversarial network. In 2017\nIEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI,\nUSA, July 21-26, 2017, pages 105\u2013114, 2017.\n\n[22] Xudong Mao, Qing Li, Haoran Xie, Raymond Y. K. Lau, Zhen Wang, and Stephen Paul Smolley.\nLeast squares generative adversarial networks. In IEEE International Conference on Computer\nVision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 2813\u20132821, 2017.\n\n[23] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization\nfor generative adversarial networks. In International Conference on Learning Representations,\n2018.\n\n[24] Wenlong Mou, Liwei Wang, Xiyu Zhai, and Kai Zheng. Generalization bounds of SGLD for\nnon-convex learning: Two theoretical viewpoints. In Conference On Learning Theory, COLT\n2018, Stockholm, Sweden, 6-9 July 2018., pages 605\u2013638, 2018.\n\n[25] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-gan: Training generative neural sam-\nplers using variational divergence minimization. In Advances in Neural Information Processing\nSystems 29: Annual Conference on Neural Information Processing Systems 2016, December\n5-10, 2016, Barcelona, Spain, pages 271\u2013279, 2016.\n\n[26] Guo-Jun Qi. Loss-sensitive generative adversarial networks on lipschitz densities. CoRR,\n\nabs/1701.06264, 2017.\n\n[27] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with\ndeep convolutional generative adversarial networks. In 4th International Conference on Learn-\ning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track\nProceedings, 2016.\n\n[28] Yunus Saatci and Andrew Wilson. Bayesian GAN.\n\nIn Advances in Neural Information\nProcessing Systems 30: Annual Conference on Neural Information Processing Systems 2017,\n4-9 December 2017, Long Beach, CA, USA, pages 3625\u20133634, 2017.\n\n[29] Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen.\nImproved techniques for training gans. In Advances in Neural Information Processing Systems\n29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016,\nBarcelona, Spain, pages 2226\u20132234, 2016.\n\n[30] Tim Salimans and Diederik P. Kingma. Weight normalization: A simple reparameterization\nto accelerate training of deep neural networks. In Advances in Neural Information Processing\nSystems 29: Annual Conference on Neural Information Processing Systems 2016, December\n5-10, 2016, Barcelona, Spain, page 901, 2016.\n\n[31] Shai Shalev-Shwartz, Ohad Shamir, Nathan Srebro, and Karthik Sridharan. Learnability,\nstability and uniform convergence. Journal of Machine Learning Research, 11:2635\u20132670,\n2010.\n\n10\n\n\f[32] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference\nattacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy,\nSP 2017, San Jose, CA, USA, May 22-26, 2017, pages 3\u201318, 2017.\n\n[33] Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. Towards demystifying\n\nmembership inference attacks. CoRR, abs/1807.09173, 2018.\n\n[34] Roman Vershynin. High-dimensional probability: An introduction with applications in data\n\nscience, volume 47. Cambridge University Press, 2018.\n\n[35] Yu-Xiang Wang, Stephen E. Fienberg, and Alexander J. Smola. Privacy for free: Posterior\nIn Proceedings of the 32nd International\nsampling and stochastic gradient monte carlo.\nConference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pages 2493\u20132502,\n2015.\n\n[36] Yu-Xiang Wang, Jing Lei, and Stephen E. Fienberg. Learning with differential privacy: Stability,\nlearnability and the suf\ufb01ciency and necessity of ERM principle. Journal of Machine Learning\nResearch, 17:183:1\u2013183:40, 2016.\n\n[37] Bingzhe Wu, Haodong Duan, Zhichao Liu, and Guangyu Sun. SRPGAN: perceptual generative\n\nadversarial network for single image super resolution. CoRR, abs/1712.05927, 2017.\n\n[38] Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine\nlearning: Analyzing the connection to over\ufb01tting. In 31st IEEE Computer Security Foundations\nSymposium, CSF 2018, Oxford, United Kingdom, July 9-12, 2018, pages 268\u2013282, 2018.\n\n[39] Han Zhang, Ian J. Goodfellow, Dimitris N. Metaxas, and Augustus Odena. Self-attention\n\ngenerative adversarial networks. CoRR, abs/1805.08318, 2018.\n\n[40] Jun-Yan Zhu, Philipp Kr\u00e4henb\u00fchl, Eli Shechtman, and Alexei A. Efros. Generative visual\nmanipulation on the natural image manifold. In Computer Vision - ECCV 2016 - 14th European\nConference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part V, pages\n597\u2013613, 2016.\n\n[41] Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image\ntranslation using cycle-consistent adversarial networks. In IEEE International Conference on\nComputer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017, pages 2242\u20132251, 2017.\n\n11\n\n\f", "award": [], "sourceid": 148, "authors": [{"given_name": "Bingzhe", "family_name": "Wu", "institution": "Peeking University"}, {"given_name": "Shiwan", "family_name": "Zhao", "institution": "IBM Research - China"}, {"given_name": "Chaochao", "family_name": "Chen", "institution": "Ant Financial"}, {"given_name": "Haoyang", "family_name": "Xu", "institution": "Peking University"}, {"given_name": "Li", "family_name": "Wang", "institution": "Ant Financial"}, {"given_name": "Xiaolu", "family_name": "Zhang", "institution": "Ant Financial Services Group"}, {"given_name": "Guangyu", "family_name": "Sun", "institution": "Peking University"}, {"given_name": "Jun", "family_name": "Zhou", "institution": "Ant Financial"}]}