{"title": "ResNets Ensemble via the Feynman-Kac Formalism to Improve Natural and Robust Accuracies", "book": "Advances in Neural Information Processing Systems", "page_first": 1657, "page_last": 1667, "abstract": "We unify the theory of optimal control of transport equations with the practice of training and testing of ResNets. Based on this unified viewpoint, we propose a simple yet effective ResNets ensemble algorithm to boost the accuracy of the robustly trained model on both clean and adversarial images. The proposed algorithm consists of two components: First, we modify the base ResNets by injecting a variance specified Gaussian noise to the output of each residual mapping. Second, we average over the production of multiple jointly trained modified ResNets to get the final prediction. These two steps give an approximation to the Feynman-Kac formula for representing the solution of a convection-diffusion equation. For the CIFAR10 benchmark, this simple algorithm leads to a robust model with a natural accuracy of {\\bf 85.62}\\% on clean images and a robust accuracy of ${\\bf 57.94 \\%}$ under the 20 iterations of the IFGSM attack, which outperforms the current state-of-the-art in defending against IFGSM attack on the CIFAR10.", "full_text": "ResNets Ensemble via the Feynman-Kac Formalism\n\nto Improve Natural and Robust Accuracies\n\nBao Wang\n\nDepartment of Mathematics\n\nUniversity of California, Los Angeles\n\nwangbaonj@gmail.com\n\nBinjie Yuan\n\nSchool of Aerospace\nTsinghua University\n\nybj14@mail.tsinghua.edu.cn\n\nZuoqiang Shi\n\nDepartment of Mathematics\n\nTsinghua University\n\nzqshi@mail.tsinghua.edu.cn\n\nStanley J. Osher\n\nDepartment of Mathematics\n\nUniversity of California, Los Angeles\n\nsjo@math.ucla.edu\n\nAbstract\n\nWe unify the theory of optimal control of transport equations with the practice\nof training and testing of ResNets. Based on this uni\ufb01ed viewpoint, we propose\na simple yet effective ResNets ensemble algorithm to boost the accuracy of the\nrobustly trained model on both clean and adversarial images. The proposed algo-\nrithm consists of two components: First, we modify the base ResNets by injecting\na variance speci\ufb01ed Gaussian noise to the output of each residual mapping, and it\nresults in a special type of neural stochastic ordinary differential equation. Second,\nwe average over the production of multiple jointly trained modi\ufb01ed ResNets to get\nthe \ufb01nal prediction. These two steps give an approximation to the Feynman-Kac\nformula for representing the solution of a convection-diffusion equation. For the\nCIFAR10 benchmark, this simple algorithm leads to a robust model with a natural\naccuracy of 85.62% on clean images and a robust accuracy of 57.94% under the\n20 iterations of the IFGSM attack, which outperforms the current state-of-the-art\nin defending against IFGSM attack on the CIFAR10. The code is available at\nhttps://github.com/BaoWangMath/EnResNet.\n\n1\n\nIntroduction\n\nDespite the extraordinary success of deep neural nets (DNNs) in image and speech recognition [23],\ntheir vulnerability to adversarial attacks raises concerns when applying them to security-critical tasks,\ne.g., autonomous cars [3, 1], robotics [14], and DNN-based malware detection systems [31, 13]. Since\nthe seminal work of Szegedy et al. [38], recent research shows that DNNs are vulnerable to many\nkinds of adversarial attacks including physical, poisoning, and inference attacks [9, 7, 30, 12, 17, 5, 4].\nThe empirical adversarial risk minimization (EARM) is one of the most successful frameworks for\nadversarial defense. Under the EARM framework, adversarial defense for (cid:96)\u221e norm based inference\nattacks can be formulated as solving the following EARM [29, 45]\n\nn(cid:88)\n\ni=1\n\nmin\nf\u2208H\n\n1\nn\n\n(1)\nwhere f (\u00b7, w) is a function in the hypothesis class H, e.g., ResNet [16], parameterized by w. Here,\n{(xi, yi)}n\ni=1 are n i.i.d. data-label pairs drawn from some high dimensional unknown distribution D,\nL(f (xi, w), yi) is the loss associated with f on (xi, yi). For classi\ufb01cation, L is typically selected to\nbe the cross-entropy. Adversarial defense for other measure based attacks can be formulated similarly.\n\nmax\ni\u2212xi(cid:107)\u221e\u2264\u0001\n\nL(f (x\n\n(cid:107)x(cid:48)\n\n(cid:48)\ni, w), yi),\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\f1.1 Our Contribution\n\nIn this work, we unify the training and testing of ResNets with the theory of transport equations\n(TEs). This uni\ufb01ed viewpoint enables us to interpret the adversarial vulnerability of ResNets as\nthe irregularity, which will be de\ufb01ned later, of the TE\u2019s solution. Based on this observation, we\npropose a new ResNets ensemble algorithm based on the Feynman-Kac formula. In a nutshell, the\nproposed algorithm consists of two essential components. First, for each l = 1, 2,\u00b7\u00b7\u00b7 , M with M\nbeing the number of residual mappings in the ResNet, we modify the l-th residual mapping from\nxl+1 = xl +F(xl) (Fig. 1 (a)) to xl+1 = xl +F(xl) + N (0, \u03c32I) (Fig. 1 (b)), where xl is the input,\nF is the residual mapping and N (0, \u03c32I) is Gaussian noise with a specially designed variance \u03c32.\nThis step can be regarded as building a simple neural stochastic differential equation. Second, we\naverage over multiple jointly trained, by solving the EARM Eq. (1), modi\ufb01ed ResNets\u2019 outputs to get\nthe \ufb01nal prediction (Fig. 1 (c)). This ensemble algorithm improves the base model\u2019s accuracy on both\nclean and adversarial data. The advantages of the proposed algorithm are summarized as follows:\n\n\u2022 It outperforms the current state-of-the-art in defending against inference attacks.\n\u2022 It improves the natural accuracy of the adversarially trained models.\n\u2022 Its defense capability can be improved dynamically as the base ResNet advances.\n\u2022 It is motivated from partial differential equation (PDE) theory, which introduces a new\nway to defend against adversarial attacks, and it is a complement to many other existing\nadversarial defenses.\n\n(a)\n\n(b)\n\n(c)\n\nFigure 1: Original (a)/noise injected (b) residual mapping. (c) Architecture of the EnResNet.\n\n1.2 Related Work\n\nThere is a massive volume of research over the last several years on defending against adversarial\nattacks for DNNs. Randomized smoothing transforms an arbitrary classi\ufb01er f into a \"smoothed\"\nsurrogate classi\ufb01er g and is certi\ufb01ably robust in (cid:96)2 norm based adversarial attacks [25, 24, 10, 43, 6,\n27, 34]. One of the ideas is to inject Gaussian noise to the input image and the classi\ufb01cation result is\nbased on the probability of the noisy image in the decision region. Our adversarial defense algorithm\ninjects noise into each residual mapping instead of the input image.\nRobust optimization for solving EARM achieves great success in defending against inference attacks\n[29, 32, 33, 44, 36, 42]. Regularization in EARM can further boost the robustness of the adversarially\ntrained models [45, 21, 35, 47]. The adversarial defense algorithms should learn a classi\ufb01er with high\ntest accuracy on both clean and adversarial data. To achieve this goal, Zhang et al. [46] developed a\nnew loss function, TRADES, that explicitly trades off between natural and robust generalization. To\nthe best our of knowledge, TRADES is the current state-of-the-art in defending against inference\nattacks on the CIFAR10. Throughout this paper, we regard TRADES as the benchmark.\nModeling DNNs as ordinary differential equations (ODEs) has drawn lots of attention recently. Chen\net al. proposed neural ODEs for deep learning [8]. E [11] modeled training ResNets as solving an\nODE optimal control problem. Haber and Ruthotto [15] constructed stable DNN architectures based\non the properties of ODEs. Lu, Zhu and et al. [28, 48] constructed novel architectures for DNNs,\nwhich were motivated from the numerical discretization schemes for ODEs.\n\n2\n\n\f1.3 Organization\n\nThis paper is organized in the following way: In section 2, we establish the connection between\ntraining/testing of ResNets and the theory of TEs. This connection gives us a way to decipher the\nadversarial vulnerability of the ResNet, and we propose a simple ensemble algorithm based on the\nFeynman-Kac formula to enhance the guaranteed adversarial robustness of ResNet. In section 3, we\ntest the ef\ufb01ciency of the proposed ResNets ensemble for adversarial defense on both CIFAR10 and\nCIFAR100. Section 4 contains some concluding remarks.\n\n2 Algorithm and Theoretical Motivation\n\n2.1 Transport Equation Modeling of ResNet\n\nThe connection between training ResNet and solving optimal control problems of the TE is in-\nvestigated in [39, 40, 26, 41]. In this section, we derive the TE model for ResNet and explain its\nadversarial vulnerability from a PDE perspective. The TE model enables us to understand the data\n\ufb02ow of the entire training and testing data in the forward and backward propagation; whereas, the\nODE models focus on the dynamics of individual data points [8].\nAs shown in Fig. 1 (a), residual mapping adds a skip connection to connect the input and output of\nthe original mapping (F), and the l-th residual mapping can be written as\n\nxl+1 = F(xl, wl) + xl,\n\n(cid:40)\n\nwith x0 = \u02c6x \u2208 T \u2282 Rd being a data point in the set T , xl and xl+1 are the input and output tensors\nof the residual mapping. The parameters wl can be learned by back-propagating the training error.\nFor \u2200 \u02c6x \u2208 T with label y, the forward propagation of ResNet can be written as\n\nl = 0, 1, . . . , L \u2212 1, with x0 = \u02c6x,\n\nxl+1 = xl + F(xl, wl),\n.\n= f (xL),\n\u02c6y\n\n(2)\nwhere \u02c6y is the predicted label, L is the number of layers, and f (x) = softmax(w0 \u00b7 x) be the output\nactivation with w0 being the trainable parameters.\nNext, let tl = l/L, for l = 0, 1,\u00b7\u00b7\u00b7 , L, with interval \u2206t = 1/L. Without considering dimensional\nconsistency, we regard xl in Eq. (2) as the value of x(t) at tl, so Eq. (2) can be rewritten as\n\nx(tl+1) = x(tl) + \u2206t \u00b7 F (x(tl), w(tl)),\n.\n\u02c6y\n= f (x(1)),\n.\n= 1\n\nwhere F\n\n\u2206tF. Eq. (3) is the forward Euler discretization of the following ODE\n\nl = 0, 1, . . . , L \u2212 1, with x(0) = \u02c6x\n\n(cid:40)\n\ndx(t)\n\ndt\n\n= F (x(t), w(t)), x(0) = \u02c6x.\n\nLet u(x, t) be a function that is constant along the trajectory de\ufb01ned by Eq. (4), then u(x, t) satis\ufb01es\n\nd\ndt\n\n(u(x(t), t)) =\n\n\u2202u\n\u2202t\n\n(x, t) + F (x, w(t)) \u00b7 \u2207u(x, t) = 0, x \u2208 Rd.\n\n(5)\n\nIf we enforce the terminal condition at t = 1 for Eq. (5) to be\n\nu(x, 1) = softmax(w0 \u00b7 x) := f (x),\n\nthen according to the fact that u(x, t) is constant along the curve de\ufb01ned by Eq. (4) (which is called\nthe characteristic curve for the TE de\ufb01ned in Eq. (5)), we have u(\u02c6x, 0) = u(x(1), 1) = f (x(1));\ntherefore, the forward propagation of ResNet for \u02c6x can be modeled as computing u(\u02c6x, 0) along the\ncharacteristic curve of the following TE\n\nMeanwhile, the backpropagation in training ResNets can be modeled as \ufb01nding the velocity \ufb01eld,\nF (x(t), w(t)), for the following control problem\n\n(3)\n\n(4)\n\n(6)\n\n(7)\n\nu(x, 1) = f (x).\n\n(cid:40) \u2202u\n\u2202t (x, t) + F (x, w(t)) \u00b7 \u2207u(x, t) = 0, x \u2208 Rd,\n\uf8f1\uf8f4\uf8f2\uf8f4\uf8f3 \u2202u\n\n\u2202t (x, t) + F (x, w(t)) \u00b7 \u2207u(x, t) = 0, x \u2208 Rd,\nu(x, 1) = f (x), x \u2208 Rd,\nu(xi, 0) = yi, xi \u2208 T, with T being the training set.\n\n3\n\n\fNote that in the above TE formulation of ResNet, u(x, 0) serves as the classi\ufb01er and the velocity\n\ufb01eld F (x, w(t)) encodes ResNet\u2019s architecture and weights. When F is very complex, u(x, 0) might\nbe highly irregular i.e. a small change in the input x can lead to a massive change in the value of\nu(x, 0). This irregular function may have a good generalizability, but it is not robust to adversarial\nattacks. Fig. 2 (a) shows a 2D illustration of u(x, 0) with the terminal condition u(x, 1) shown in\nFig. 2 (d); we will discuss this in detail later in this section.\n\n(a) \u03c3 = 0\n\n(b) \u03c3 = 0.01\n\n(c) \u03c3 = 0.1\n\n(d) u(x, 1)\n\nFigure 2: (d): terminal condition for Eq. (8); (a), (b), and (c): solutions of the convection-diffusion\nequation, Eq. (8), at t = 0 with different diffusion coef\ufb01cients \u03c3.\n\n2.2 Adversarial Defense by ResNets Ensemble via the Feynman-Kac Formalism\n\n(cid:40) \u2202u\n\u2202t (x, t) + F (x, w(t)) \u00b7 \u2207u(x, t) + 1\n\nUsing a speci\ufb01c level set of u(x, 0) in Fig. 2 (a) for classi\ufb01cation suffers from adversarial vulnerability:\nA tiny perturbation in x will lead the output to go across the level set, thus leading to misclassi\ufb01cation.\nTo mitigate this issue, we introduce a diffusion term 1\n2 \u03c32\u2206u to Eq. (6), with \u03c3 being the diffusion\ncoef\ufb01cient and \u2206 is the Laplace operator in Rd, to make the level sets of the TE more regular. This\nimproves robustness of the classi\ufb01er. Hence, we arrive at the following convection-diffusion equation\n\n2 \u03c32\u2206u(x, t) = 0, x \u2208 Rd,\n\nt \u2208 [0, 1),\n\nu(x, 1) = f (x).\n\n(8)\nThe solution to Eq. (8) is much more regular when \u03c3 (cid:54)= 0 than when \u03c3 = 0. We consider the\nsolution of Eq. (8) in a 2D unit square with periodic boundary conditions, and on each grid point of\nthe mesh the velocity \ufb01eld F (x, w(t)) is a random number sampled uniformly from \u22121 to 1. The\nterminal condition is also randomly generated, as shown in Fig. 2 (d). This 2D convection-diffusion\nequation is solved by the pseudo-spectral method with spatial and temporal step sizes being 1/128\nand 1 \u00d7 10\u22123, respectively. Figure 2 (a), (b), and (c) illustrate the solutions when \u03c3 = 0, 0.01, and\n0.1, respectively. These show that as \u03c3 increases, the solution becomes more regular, which makes\nthe classi\ufb01er more robust, but might be less accurate on clean data. The \u03c3 should be selected to\nhave a good trade-off between accuracy and robustness. Moreover, we have the following theoretical\nguarantee for robustness of the solution to the convection-diffusion equation.\nTheorem 1. [22] Let F (x, t) be a Lipschitz function in both x and t, and f (x) be a bounded function.\nConsider the following initial value problem of the convection-diffusion equation (\u03c3 (cid:54)= 0)\n\n(cid:40) \u2202u\n\u2202t (x, t) + F (x, w(t)) \u00b7 \u2207u(x, t) + 1\n\nu(x, 1) = f (x).\n\n2 \u03c32\u2206u(x, t) = 0, x \u2208 Rd,\n\nt \u2208 [0, 1),\n\n(9)\n\n(cid:16)(cid:107)\u03b4(cid:107)2\n\n(cid:17)\u03b1\n\nThen, for any small perturbation \u03b4, we have |u(x + \u03b4, 0)\u2212 u(x, 0)| \u2264 C\nfor some constant\n\u03b1 > 0 if \u03c3 \u2264 1. Here, (cid:107)\u03b4(cid:107)2 is the (cid:96)2 norm of \u03b4, and C is a constant that depends on d, (cid:107)f(cid:107)\u221e, and\n(cid:107)F(cid:107)L\u221e\n\n\u03c3\n\n.\n\nx,t\n\nAccording to the above observation, instead of using u(x, 0) of the TE\u2019s solution for classi\ufb01cation,\nwe use that of the convection-diffusion equation. The above convection-diffusion equation can be\nsolved using the Feynman-Kac formula [18] in high dimensional space, which gives u(\u02c6x, 0) as 1\n\nu(\u02c6x, 0) = E [f (x(1))|x(0) = \u02c6x] ,\n\n(10)\n\nwhere x(t) is an It\u00f4 process,\n\n1A detailed derivation is available in the supplementary material.\n\ndx(t) = F (x(t), w(t))dt + \u03c3dBt,\n\n4\n\n\fGaussian noise \u03c3N (0, I), where \u03c3 = a(cid:112)Var(xl + F(xl)) with a being a tunable parameter, to\n\nand u(\u02c6x, 0) is the conditional expectation of f (x(1)).\nWe approximate the Feynman-Kac formula by an ensemble of modi\ufb01ed ResNets in the following\nway: Accoding to the Euler\u2013Maruyama method [2], the term \u03c3dBt can be approximated by adding\neach residual mapping xl+1 = xl + F(xl). This gives the modi\ufb01ed residual mapping xl+1 =\nxl + F(xl) + \u03c3N (0, I), as illustrated in Fig. 1 (b). Let ResNet\u2019 denote the modi\ufb01ed ResNet where\nwe inject noise to each residual mapping of the original ResNet. In a nutshell, ResNet\u2019s approximation\nto the Feynman-Kac formula is an ensemble of jointly trained ResNet\u2019 as illustrated in Fig. 1 (c). 2\nWe call this ensemble of ResNets as EnResNet. For instance, an ensemble of n ResNet20 is denoted\nas EnnResNet20.\n\n2.3 Robust Training of the EnResNet\n\nWe use the PGD adversarial training [29] to robustly train EnResNets with \u03c3 = 0.1 on both CIFAR10\nand CIFAR100 [20] with standard data augmentation [16]. The attack in the PGD adversarial training\nis merely iterative fast gradient sign method (IFGSM) with an initial random perturbation on the\nclean data. Other methods to solve EARM can also be used to train EnResNets. All computations are\ncarried out on a machine with a single Nvidia Titan Xp graphics card.\n\n2.4 Attack Methods\n\nWe attack the trained model, f (x, w), by (cid:96)\u221e norm based untargeted FGSM, IFGSM [12], and C&W\n[7] attacks in both white-box and blind fashions. In blind attacks, we use the target model to classify\nthe adversarial images crafted by attacking the oracle model in a white-box approach. For a given\ninstance (x, y):\n\u2022 FGSM searches the adversarial\n\n.\n=\nL(f (x(cid:48), w), y), subject to the constraint ||x(cid:48) \u2212 x||\u221e \u2264 \u0001 with \u0001 being the maximum pertur-\nbation. For the linearized loss function, L(x(cid:48), y) \u2248 L(x, y) + \u2207xL(x, y)T \u00b7 (x(cid:48) \u2212 x), the optimal\nadversarial is\n\nimage x(cid:48) by maximizing the loss function L(x(cid:48), y)\n\n(11)\n\u2022 IFGSM, Eq. (12), iterates FGSM with step size \u03b1 and clips the perturbed image to generate the\n\nx(cid:48) = x + \u0001 \u00b7 sign (\u2207xL(x, y)) .\n\nenhanced adversarial attack, with x(0) being the clean data,\n\nx(m) = Clipx,\u0001{x(m\u22121) + \u03b1 \u00b7 sign(\u2207xL(x(m\u22121), y))}.\n\n\u2022 C&W attack searches the targeted adversarial image by solving\n\n||\u03b4||\u221e, subject to f (w, x + \u03b4) = t, x + \u03b4 \u2208 [0, 1]d,\n\nmin\n\n\u03b4\n\n(12)\n\n(13)\n\nwhere \u03b4 is the adversarial perturbation and t is the target label. Carlini et al. [7] proposed the\nfollowing approximation to Eq. (13),\n\n(cid:26)\n\n(cid:27)\n\n|| 1\n2\n\n(tanh(u) + 1) \u2212 x||\u221e + c \u00b7 max\n\nu\n\nmin\n\n,\n(14)\nwhere Z(\u00b7) is the logit vector for the input, i.e., the output of the DNN before the softmax layer.\nThis unconstrained optimization can be solved ef\ufb01ciently by using the Adam optimizer [19].\n\n(tanh(u)) + 1)t\n\n(Z(\n\n\u2212\u03ba, max\ni(cid:54)=t\n\n1\n2\n\n(tanh(u)) + 1)i) \u2212 Z(\n\n1\n2\n\nIn the following experiments, we set \u0001 = 8/255 in both FGSM and IFGSM attacks. Additionally,\nin IFGSM we set m = 20 and \u03b1 = 2/255, and denote it as IFGSM20. For C&W attack, we run 50\niterations of Adam with learning rate 6 \u00d7 10\u22124 and set c = 10 and \u03ba = 0.\n\n3 Numerical Results\n\nIn this section, we numerically verify that the robustly trained EnResNets are more accurate, on both\nclean and adversarial data, than robustly trained ResNets and ensemble of ResNets without noise\n\n2To ease the notation, in what follows, we use ResNet in place of ResNet\u2019 when there is no ambiguity.\n\n5\n\n\finjection 3. To avoid the gradient mask issue of EnResNets due to the injected noise, we use the\nExpectation over Transformation (EOT) strategy [5] to compute the gradient which is averaged over\n\ufb01ve independent runs.\n\n3.1 EnResNets\n\nIn robust training, we run 200 epochs of the PGD adversarial training (10 iterations of IFGSM with\n\u03b1 = 2/255 and \u0001 = 8/255, and an initial random perturbation of magnitude \u0001) with initial learning\nrate 0.1, which decays by a factor of 10 at the 80th, 120th, and 160th epochs. The training data is\nsplit into 45K/5K for training and validation, the model with the best validation accuracy is used for\ntesting. En1ResNet20 denotes the ensemble of only one ResNet20 which is merely adding noise to\neach residual mapping, and similar notations apply to other DNNs.\nFirst, consider natural (Anat) and robust (Arob) accuracies of the PGD adversarially trained models\non the CIFAR10, where Anat and Arob are measured on clean and adversarial images, respectively.\nAll results are listed in Table 1. The robustly trained ResNet20 has accuracies 50.89%, 46.03%\n(close to that reported in [29]), and 58.73%, respectively, under the FGSM, IFGSM20, and C&W\nattacks. Moreover, it has a natural accuracy of 75.11%. En5ResNet20 boosts natural accuracy\nto 82.52%, and improves the corresponding robust accuracies to 58.92%, 51.48%, and 67.73%,\nrespectively. Simply injecting noise to each residual mapping of ResNet20 can increase Anat by\n\u223c 2% and Arob by \u223c 3% under the IFGSM20 attack. The advantages of EnResNets are also veri\ufb01ed\nby experiments on ResNet44, ResNet110, and their ensembles. Note that ensemble of high capacity\nResNet is more robust than low capacity model: as shown in Table 1, En2ResNet110 is more accurate\nthan En2ResNet44 which in turn is more accurate than En2ResNet20 in classifying both clean and\nadversarial images. The robustly trained En1WideResNet34-10 has 86.19% and 56.60%, respectively,\nnatural and robust accuracies under the IFGSM20 attack. Compared with the current state-of-the-art\n[46], En1WideResNet34-10 has almost the same robust accuracy (56.60% v.s. 56.61%) under the\nIFGSM20 attack but better natural accuracy (86.19% v.s. 84.92%). Figure 3 plots the evolution of\ntraining and validation accuracies of ResNet20 and ResNet44 and their different ensembles.\n\nFigure 3: Evolution of training and validation accuracies. Left: ResNet20 and different ensembles of\nnoise injected ResNet20. Right: ResNet44 and different ensembles of noise injected ResNet44.\n\nSecond, consider the robustly trained models under blind attacks. In this scenario, we use the target\nmodel to classify the adversarial images crafted by applying FGSM, IFGSM20, and C&W attacks\nto the oracle model. As listed in Table 2, EnResNets are always more robust than the base ResNets\nunder different blind attacks. For instance, when En5ResNet20 is used to classify adversarial images\ncrafted by attacking ResNet20 with FGSM, IFGSM20, and C&W attacks, the accuracies are 64.07%,\n62.99%, and 76.57%, respectively. Conversely, the accuracies of ResNet20 are only 61.69%, 58.74%,\nand 73.77%, respectively.\nThird, we perform experiments on the CIFAR100 to further verify the ef\ufb01ciency of EnResNets\nin defending against adversarial attacks. Table 3 lists natural and robust accuracies of ResNet20,\nResNet44, and their ensembles under white-box attacks. The robust accuracy under the blind attacks\nis listed in Table 4. The natural accuracy of the PGD adversarially trained baseline ResNet20 is\n46.02%, and it has robust accuracies 24.77%, 23.23%, and 32.42% under FGSM, IFGSM20, and\n\n3The baseline ResNet implementation is available at https://github.com/akamaster/pytorch_\n\nresnet_cifar10/blob/master/resnet.py\n\n6\n\n\fTable 1: Natural and robust accuracies of different base and noise injected ensembles of robustly\ntrained ResNets on the CIFAR10. Unit: %.\n\nModel\n\nResNet20\n\nEn1ResNet20\nEn2ResNet20\nEn5ResNet20\n\nResNet44\n\nEn1ResNet44\nEn2ResNet44\nResNet110\n\nEn2ResNet110\n\nEn1WideResNet34-10\n\ndataset\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\n\nAnat\n75.11\n77.21\n80.34\n82.52\n78.89\n82.03\n82.91\n82.19\n82.43\n86.19\n\nArob (FGSM)\n\nArob (IFGSM20)\n\nArob (C&W)\n\n50.89\n55.35\n57.23\n58.92\n54.54\n57.80\n58.29\n57.61\n59.24\n61.82\n\n46.03\n49.06\n50.06\n51.48\n48.85\n51.83\n51.86\n52.02\n53.03\n56.60\n\n58.73\n65.69\n66.47\n67.73\n61.33\n66.00\n66.89\n62.92\n68.67\n69.32\n\nTable 2: Accuracies of robustly trained models on adversarial images of CIFAR10 crafted by\nattacking the oracle model with different attacks. Unit: %.\n\nModel\nResNet20\n\nEn5ResNet20\n\nResNet44\n\nEn2ResNet44\nResNet110\n\nEn2ResNet110\n\ndataset\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\n\nOracle\n\nArob (FGSM)\n\nArob (IFGSM20)\n\nArob (C&W)\n\nEn5ResNet20\n\nResNet20\n\nEn2ResNet44\n\nResNet44\n\nEn2ResNet110\n\nResNet110\n\n61.69\n64.07\n63.87\n64.52\n64.19\n66.26\n\n58.74\n62.99\n60.66\n61.23\n61.80\n62.89\n\n73.77\n76.57\n75.83\n76.99\n75.19\n77.71\n\nC&W attacks, respectively. En5ResNet20 increases them to 51.72%, 31.64%, 27.80%, and 40.44%,\nrespectively. The ensemble of ResNets is more effective in defending against adversarial attacks than\nmaking the ResNets deeper. For instance, En2ResNet20 that has \u223c 0.27M \u00d7 2 parameters is much\nmore robust to adversarial attacks, FGSM (30.20% v.s. 28.40%), IFGSM20 (26.25% v.s. 25.81%),\nand C&W (40.06% v.s. 36.06%), than ResNet44 with \u223c 0.66M parameters. Under blind attacks,\nEn2ResNet20 is also signi\ufb01cantly more robust to different attacks where the opponent model is used\nto generate adversarial images. Under the same model and computation complexity, EnResNets is\nmore robust to adversarials and more accurate on clean images than deeper nets.\n\nTable 3: Natural and robust accuracies of robustly trained ResNet20 and different ensemble of\nnoise injected ResNet20 on the CIFAR100. Unit: %.\n\nModel\nResNet20\n\nEn2ResNet20\nEn5ResNet20\n\nResNet44\n\ndataset\n\nCIFAR100\nCIFAR100\nCIFAR100\nCIFAR100\n\nAnat\n46.02\n50.68\n51.72\n50.38\n\nArob (FGSM)\n\nArob (IFGSM20)\n\nArob (C&W)\n\n24.77\n30.20\n31.64\n28.40\n\n23.23\n26.25\n27.80\n25.81\n\n32.42\n40.06\n40.44\n36.06\n\nTable 4: Accuracies of robustly trained models on the adversarial images of CIFAR100 crafted\nby attacking the oracle model with different attacks. Unit: %.\n\nModel\nResNet20\n\nEn2ResNet20\n\ndataset\n\nOracle\n\nCIFAR100\nCIFAR100\n\nEn2ResNet20\n\nResNet20\n\nArob (FGSM)\n\nArob (IFGSM20)\n\nArob (C&W)\n\n33.08\n34.15\n\n30.79\n33.34\n\n41.52\n48.21\n\nFigure 4 depicts a few selected images from the CIFAR10 and their adversarial ones crafted by apply-\ning either IFGSM20 or C&W attack to attack both ResNet20 and En5ResNet20. Both adversarially\ntrained ResNet20 and En5ResNet20 fail to correctly classify any of the adversarial versions of these\nfour images. For the deer image, both ResNet and En5ResNet have only slightly higher con\ufb01dence\nto classify them as a deer than as a horse, and it might also be dif\ufb01cult for a human to distinguish it\nfrom a horse.\n\n7\n\n\fFigure 4: Column 1: original images and labels; column 2-3 (4-5): adversarial images crafted by\nusing IFGSM20 and C&W to attack ResNet20 (En5ResNet20) and corresponding predicted labels.\n\n3.2\n\nIntegration of Separately Trained EnResNets\n\nIn the previous subsection, we veri\ufb01ed the adversarial defense capability of EnResNet, which is\nan approximation to the Feynman-Kac formula to solve the convection-diffusion equation. As we\nshowed, when more ResNets and larger models are involved in the ensemble, both natural and robust\naccuracies are improved. However, EnResNet proposed above requires to train the ensemble jointly,\nwhich poses memory challenges for training ultra-large ensembles. To overcome this issue, we\nconsider training each component of the ensemble individually and integrating them together for\nprediction. The major bene\ufb01t of this strategy is that with the same amount of GPU memory, we can\ntrain a much larger model for inference since the batch size used in inference can be one.\nTable 5 lists natural and robust accuracies of the integration of separately trained EnResNets on the\nCIFAR10. The integration have better robust accuracy than each component. For instance, the integra-\ntion of En2ResNet110 and En1WideResNet34-10 gives a robust accuracy 57.94% under the IFGSM20\nattack, which is remarkably better than both En2ResNet110 (53.05%) and En1WideResNet34-10\n(56.60%). To the best of our knowledge, 57.94% outperforms the current state-of-the-art [46] by\n1.33%. The effectiveness of the integration of separately trained EnResNets sheds light on the\ndevelopment of ultra-large models to improve ef\ufb01ciency for adversarial defense.\n\nTable 5: Natural and robust accuracies of different integration of different robustly trained EnResNets\non the CIFAR10. Unit: %.\n\nModel\n\nEn2ResNet20&En5ResNet20\nEn2ResNet44&En5ResNet20\nEn2ResNet110&En5ResNet20\n\nEn2ResNet110&En1WideResNet34-10\n\ndataset\nCIFAR10\nCIFAR10\nCIFAR10\nCIFAR10\n\nAnat Arob (FGSM) Arob (IFGSM20) Arob (C&W)\n82.82\n82.99\n83.57\n85.62\n\n59.14\n59.64\n60.63\n62.48\n\n53.15\n53.86\n54.87\n57.94\n\n68.00\n69.36\n70.02\n70.20\n\n3.3 Gradient Mask and Comparison with Simple Ensembles\n\nBesides applying EOT gradient, we further verify that our defense is not due to obfuscated gradi-\nent. We use IFGSM20 to attack naturally trained (using the same approach as that used in [16])\nEn1ResNet20, En2ResNet20, and En5ResNet20, and the corresponding accuracies are: 0%, 0.02%,\nand 0.03%, respectively. All naturally trained EnResNets are easily fooled by IFGSM20, thus gradient\nmask does not play an important role in EnResNets for adversarial defense [4]. However, under the\nFGSM attack with \u0001 = 8/255, the naturally trained En1ResNet20 and En2ResNet20 (with injected\nGaussian noise of standard deviation 0.1) has robust accuracies 27.93% and 28.75%, resp., and it is\nsigni\ufb01cantly higher than that of the naturally trained ResNet20. These results show that the naturally\ntrained EnResNets are also more resistant to adversarial attacks.\nEnsemble of models for adversarial defense has been studied in [37]. Here, we show that ensembles\nof robustly trained ResNets without noise injection cannot boost natural and robust accuracy much.\nThe natural accuracy of jointly (separately) adversarially trained ensemble of two ResNet20 without\nnoise injection is 75.75% (74.96%), which does not substantially outperform ResNet20 with a natural\n\n8\n\n\faccuracy 75.11%. The corresponding robust accuracies are 51.11% (51.68%), 47.28% (47.86%),\nand 59.73% (59.80%), respectively, under the FGSM, IFGSM20, and C&W attacks. These robust\naccuracies are much inferior to that of En2ResNet20. Furthermore, the ensemble of separately trained\nrobust ResNet20 and robust ResNet44 gives a natural accuracy of 77.92%, and robust accuracies\nare 54.73%, 51.47%, 61.77% under the above three attacks. These results reveal that ensemble\nadversarially trained ResNets via the Feynman-Kac formalism is much more accurate than standard\nensemble in both natural and robust generalizations.\n\n3.4 Memory and Inference Time Consumption\n\nEnResNets has negligible overhead in inference time and memory compared with inference by the\nstandard ResNet. For instance, the inference time of ResNet20 and En1ResNet20 for CIFAR10\nclassi\ufb01cation, averaged over 100 runs, with batch size 1K on a Titan Xp are 1.6941s and 1.6943s,\nresp. The corresponding peak memory is 4807MB for both ResNet20 and En1ResNet20.\n\n3.5 Sensitivity to the Noise Injection\n\nNow, we consider the effects of the injected Gaussian noise, with standard deviation \u03c3, Table 6 lists\nArob (IFGSM20) of the robustly trained En2ResNet20 with different \u03c3. 0.1 gives a good trade-off\nbetween accuracy and variance.\n\nTable 6: Robust accuracy of the PGD adversarially trained En2ResNet20 with different Gaussian\nnoise injection. (\ufb01ve runs)\n\n\u03c3\n\nArob (IFGSM20)\n\n0.05\n\n0.1\n\n50.05% \u00b1 0.27% 50.06% \u00b1 0.35% 50.51% \u00b1 0.90% 43.51% \u00b1 3.78%\n\n0.4\n\n0.8\n\n3.6 Sharing Weights Ensemble\n\nFinally, we need to point out that the direct ResNet ensemble counterpart of the Feynman-Kac\nformalism needs to share weights. Table 7 shows that, the share weights ensemble (SWE) also\nimproves both natural and robust accuracies which veri\ufb01es the ef\ufb01cacy of our PDE formalism.\nMoreover, to further improve the ensemble model\u2019s performance, we generalize SWE to non-share\nweights ensemble (NSWE) with the consideration of increasing the model capacity.\nTable 7: Accuracy of the robustly trained n\u00d7En1ResNet20 which denotes the ensemble of n\nshare-weights En1ResNet20.\n\nAnat\n\nArob (IFGSM20)\n\nResNet20\n75.11%\n46.03%\n\n1\u00d7 En1ResNet20\n\n2\u00d7 En1ResNet20\n\n5\u00d7 En1ResNet20\n\n77.21%\n49.06%\n\n77.88%\n49.17%\n\n77.99%\n49.20%\n\n4 Conclusions\n\nIn this paper, we utilize a transport equation to model ResNets\u2019 data \ufb02ow. The lack of regularity of the\ntransport equation\u2019s solution explains ResNets\u2019 adversarial vulnerability. The analogy of regularizing\nthe solution of transport equation by adding a diffusion term motivates us to propose a ResNets\nensemble based on the Feynman-Kac formula. The adversarially trained EnResNet remarkably\nimproves both natural and robust accuracies towards adversarial attacks. Our method is a complement\nto many existing adversarial defense algorithms, for instance, directly replacing the cross-entropy\nloss with the TRADES loss [46] can further improve the robust accuracy, under the IFGSM20 attack,\nof the WideResNet used above by \u223c 0.9%. As a future work, we propose to combine EnResNet with\nthe surrogate loss function design and regularization [45].\n\nAcknowledgments\n\nBao Wang thanks Dr. Jiajun Tong and Dr. Yuming Zhang for stimulating discussion on the stability\ntheorem for the convection-diffusion equation.\n\n9\n\n\fReferences\n[1] Adversarial machine learning against Tesla\u2019s autopilot. https://www.schneier.com/blog/archives/\n\n2019/04/adversarial_mac.html.\n\n[2] P. Kloeden abd E. Platen. Numerical Solution of Stochastic Differential Equations. Springer, 1992.\n\n[3] N. Akhtar and A. Mian. Threat of adversarial attacks on deep learning in computer vision: A survey. arXiv\n\npreprint arXiv:1801.00553, 2018.\n\n[4] A. Athalye, N. Carlini, and D. Wagner. Obfuscated gradients give a false sense of security: Circumventing\n\ndefenses to adversarial examples. International Conference on Machine Learning, 2018.\n\n[5] A. Athalye, L. Engstrom, A. Ilyas, and K. Kwok. Synthesizing robust adversarial examples. International\n\nConference on Machine Learning, 2018.\n\n[6] X. Cao and N. Gong. Mitigating evasion attacks to deep neural networks via region-based classi\ufb01cation.\n\nIn 33rd Annual Computer Security Applications Conference, 2017.\n\n[7] N. Carlini and D.A. Wagner. Towards evaluating the robustness of neural networks. IEEE European\n\nSymposium on Security and Privacy, pages 39\u201357, 2016.\n\n[8] R. Chen, Y. Rubanova, J. Bettencourt, and D. Duvenaud. Neural ordinary differential equations. In\n\nAdvances in Neural Information Processing Systems, 2018.\n\n[9] X. Chen, C. Liu, B. Li, K. Liu, and D. Song. Targeted backdoor attacks on deep learning systems using\n\ndata poisoning. arXiv preprint arXiv:1712.05526, 2017.\n\n[10] J. Cohen, E. Rosenfeld, and J.Z. Kolter. Certi\ufb01ed adversarial robustness via randomized smoothing. arXiv\n\npreprint arXiv:1902.02918v1, 2019.\n\n[11] W. E. A proposal on machine learning via dynamical systems. Communications in Mathematics and\n\nStatistics, 5:1\u201311, 2017.\n\n[12] I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. arXiv\n\npreprint arXiv:1412.6275, 2014.\n\n[13] K. Grosse, N. Papernot, P. Manoharan, M. Backes, and P. McDaniel. Adversarial perturbations against\n\ndeep neural networks for malware classi\ufb01cation. arXiv preprint arXiv:1606.04435, 2016.\n\n[14] A. Guisti, J. Guzzi, D.C. Ciresan, F.L. He, J.P. Rodriguez, F. Fontana, M. Faessler, C. Forster, J. Schmidhu-\nber, G. Di Carlo, and et al. A machine learning approach to visual perception of forecast trails for mobile\nrobots. IEEE Robotics and Automation Letters, pages 661\u2013667, 2016.\n\n[15] E. Haber and L. Ruthotto. Stable architectures for deep neural networks. Inverse Problems, 34:014004,\n\n2017.\n\n[16] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In CVPR, pages\n\n770\u2013778, 2016.\n\n[17] A. Ilyas, L. Engstrom, A. Athalye, and J. Lin. Black-box adversarial attacks with limited queries and\n\ninformation. International Conference on Machine Learning, 2018.\n\n[18] M. Kac. On distributions of certain Wiener functionals. Transactions of the American Mathematical\n\nSociety, 65:1\u201313, 1949.\n\n[19] D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.\n\n[20] A. Krizhevsky. Learning multiple layers of features from tiny images. 2009.\n\n[21] A. Kurakin, I. Goodfellow, and S. Bengio. Adversarial machine learning at scale.\n\nConference on Learning Representations, 2017.\n\nIn International\n\n[22] O. Ladyzhenskaja, V. Solonnikov, and N. Uraltseva. Linear and Quasilinear Equations of Parabolic Type.\n\nAmerican Mathematical Society, Providence, R.I., 1968.\n\n[23] Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 521:436\u2013444, 2015.\n\n[24] M. Lecuyer, V. Atlidakis, R. Geambasu, D. Hsu, and S. Jana. Certi\ufb01ed robustness to adversarial examples\n\nwith differential privacy. In IEEE Symposium on Security and Privacy (SP), 2019.\n\n10\n\n\f[25] B. Li, C. Chen, W. Wang, and L. Carin. Second-order adversarial attack and certi\ufb01able robustness. arXiv\n\npreprint arXiv:1809.03113, 2018.\n\n[26] Z. Li and Z. Shi. Deep residual learning and pdes on manifold. arXiv preprint arXiv:1708:05115, 2017.\n\n[27] X. Liu, M. Cheng, H. Zhang, and C. Hsieh. Towards robust neural networks via random self-ensemble. In\n\nProceedings of the European Conference on Computer Vision (ECCV), pages 369\u2013385, 2018.\n\n[28] Y. Lu, A. Zhong, Q. Li, and Bin Dong. Beyond \ufb01nite layer neural networks: Bridging deep architectures\n\nand numerical differential equations. International Conference on Machine Learning, 2018.\n\n[29] A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu. Towards deep learning models resistant to\n\nadversarial attacks. In International Conference on Learning Representations, 2018.\n\n[30] N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z.B. Celik, and A. Swami. The limitations of deep\nlearning in adversarial settings. IEEE European Symposium on Security and Privacy, pages 372\u2013387,\n2016.\n\n[31] N. Papernot, P. McDaniel, A. Sinha, and M. Wellman. Sok: Towards the science of security and privacy in\n\nmachien learning. arXiv preprint arXiv:1611.03814, 2016.\n\n[32] A. Raghunathan, J. Steinhardt, and P. Liang. Certi\ufb01ed defenses against adversarial examples. In Interna-\n\ntional Conference on Learning Representations, 2018.\n\n[33] A. Raghunathan, J. Steinhardt, and P. Liang. Semide\ufb01nite relaxations for certifying robustness to adversarial\n\nexamples. In Advances in Neural Information Processing Systems, 2018.\n\n[34] A. Rakin, Z. He, and D. Fan. Parametric noise injection: Trainable randomness to improve deep neural\n\nnetwork robustness against adversarial attack. arXiv preprint arXiv:1811.09310, 2018.\n\n[35] A. Ross and F. Doshi-Velez. Improving the adversarial robustness and interpretability of deep neural\n\nnetworks by regularizing their input gradients. arXiv preprint arXiv:1711.09404, 2017.\n\n[36] H. Salman, G. Yang, H. Zhang, C. Hsieh, and P. Zhang. A convex relaxation barrier to tight robustness\n\nveri\ufb01cation of neural networks. arXiv preprint arXiv:1902.08722, 2019.\n\n[37] T. Strauss, M. Hanselmann, A. Junginger, and H. Ulmer. Ensemble methods as a defense to adversarial\n\nperturbations against deep neural networks. arXiv preprint arXiv:1709.0342, 2017.\n\n[38] C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing\n\nproperties of neural networks. arXiv preprint arXiv:1312.6199, 2013.\n\n[39] B. Wang, A. T. Lin, Z. Shi, W. Zhu, P. Yin, A. L. Bertozzi, and S. J. Osher. Adversarial defense via data\ndependent activation function and total variation minimization. arXiv preprint arXiv:1809.08516, 2018.\n\n[40] B. Wang, X. Luo, Z. Li, W. Zhu, Z. Shi, and S. Osher. Deep neural nets with interpolating function as\n\noutput activation. Advances in Neural Information Processing Systems, 2018.\n\n[41] B. Wang and S. Osher. Graph Interpolating Activation Improves Both Natural and Robust Accuracies in\n\nData-Ef\ufb01cient Deep Learning. arXiv e-prints, page arXiv:1907.06800, Jul 2019.\n\n[42] Y. Wang, X. Ma, J. Bailey, J. Yi, B. Zhu, and Q. Gu. On the convergence and robustness of adversarial\n\ntraining. International Conference on Machine Learning, 2019.\n\n[43] E. Wong and J. Kolter. Provable defenses against adversarial examples via the convex outer adversarial\n\npolytope. In International Conference on Machine Learning, 2018.\n\n[44] E. Wong, F. Schmidt, J. Metzen, and J. Kolter. Scaling provable adversarial defenses. In Advances in\n\nNeural Information Processing Systems, 2018.\n\n[45] D. Yin, K. Ramchandran, and P. Bartlett. Rademacher complexity for adversarially robust generalization.\n\narXiv preprint arXiv:1810.11914, 2018.\n\n[46] H. Zhang, Y. Yu, J. Jiao, E. Xing, L. Ghaoui, and M. Jordan. Theoretically principled trade-off between\n\nrobustness and accuracy. arXiv preprint arXiv:1901.08573, 2019.\n\n[47] S. Zheng, Y. Song, T. Leung, and I. Goodfellow. Improving the robustness of deep neural networks via\n\nstability training. In IEEE Conference on Computer Vision and Pattern Recognition, 2016.\n\n[48] M. Zhu, B. Chang, and C. Fu. Convolutional neural networks combined with runge-kutta methods. arXiv\n\npreprint arXiv:1802.08831, 2018.\n\n11\n\n\f", "award": [], "sourceid": 932, "authors": [{"given_name": "Bao", "family_name": "Wang", "institution": "UCLA"}, {"given_name": "Zuoqiang", "family_name": "Shi", "institution": "zqshi@mail.tsinghua.edu.cn"}, {"given_name": "Stanley", "family_name": "Osher", "institution": "UCLA"}]}