{"title": "Learning from brains how to regularize machines", "book": "Advances in Neural Information Processing Systems", "page_first": 9529, "page_last": 9539, "abstract": "Despite impressive performance on numerous visual tasks, Convolutional Neural Networks (CNNs) --- unlike brains --- are often highly sensitive to small perturbations of their input, e.g. adversarial noise leading to erroneous decisions. We propose to regularize CNNs using large-scale neuroscience data to learn more robust neural features in terms of representational similarity. We presented natural images to mice and measured the responses of thousands of neurons from cortical visual areas. Next, we denoised the notoriously variable neural activity using strong predictive models trained on this large corpus of responses from the mouse visual system, and calculated the representational similarity for millions of pairs of images from the model's predictions. We then used the neural representation similarity to regularize CNNs trained on image classification by penalizing intermediate representations that deviated from neural ones. This preserved performance of baseline models when classifying images under standard benchmarks, while maintaining substantially higher performance compared to baseline or control models when classifying noisy images. Moreover, the models regularized with cortical representations also improved model robustness in terms of adversarial attacks. This demonstrates that regularizing with neural data can be an effective tool to create an inductive bias towards more robust inference.", "full_text": "Learning From Brains How to Regularize Machines\n\nZhe Li1-2,*, Wieland Brendel4-5, Edgar Y. Walker1-2,7, Erick Cobos1-2,\n\nTaliah Muhammad1-2, Jacob Reimer1-2, Matthias Bethge4-6,\n\nFabian H. Sinz2,5,7, Xaq Pitkow1-3, Andreas S. Tolias1-3\n\n1 Department of Neuroscience, Baylor College of Medicine\n\n2 Center for Neuroscience and Arti\ufb01cial Intelligence, Baylor College of Medicine\n\n3 Department of Electrical and Computer Engineering, Rice University\n\n4 Centre for Integrative Neuroscience, University of T\u00fcbingen\n\n5 Bernstein Center for Computational Neuroscience, University of T\u00fcbingen\n\n6 Institute for Theoretical Physics, University of T\u00fcbingen\n\n7 Institute Bioinformatics and Medical Informatics, University of T\u00fcbingen\n\n*zhel@bcm.edu\n\nAbstract\n\nDespite impressive performance on numerous visual tasks, Convolutional Neural\nNetworks (CNNs) \u2014 unlike brains \u2014 are often highly sensitive to small pertur-\nbations of their input, e.g. adversarial noise leading to erroneous decisions. We\npropose to regularize CNNs using large-scale neuroscience data to learn more\nrobust neural features in terms of representational similarity. We presented natural\nimages to mice and measured the responses of thousands of neurons from cortical\nvisual areas. Next, we denoised the notoriously variable neural activity using strong\npredictive models trained on this large corpus of responses from the mouse visual\nsystem, and calculated the representational similarity for millions of pairs of images\nfrom the model\u2019s predictions. We then used the neural representation similarity\nto regularize CNNs trained on image classi\ufb01cation by penalizing intermediate\nrepresentations that deviated from neural ones. This preserved performance of\nbaseline models when classifying images under standard benchmarks, while main-\ntaining substantially higher performance compared to baseline or control models\nwhen classifying noisy images. Moreover, the models regularized with cortical\nrepresentations also improved model robustness in terms of adversarial attacks.\nThis demonstrates that regularizing with neural data can be an effective tool to\ncreate an inductive bias towards more robust inference.\n\n1\n\nIntroduction\n\nConvolutional neural network (CNN) models are widely used in computer vision tasks, and can\nachieve super-human performance on many classi\ufb01cation tasks [1, 2]. However, there is a still huge\ngap between these models and the human visual system in terms of robustness and generalization\n[3, 4, 5]. In fact, the invariant neural representations and the ability to generalize across complex\ntransformations has been seen as the hallmark of visual intelligence [6, 7, 8, 9]. Understanding why\nthe visual system has superior performance on so many perceptual problems is one of the central\nquestions of neuroscience and machine learning. In particular, CNNs are vulnerable to adversarial\nattacks and noise distortions [10, 3, 4] while human perception is barely affected by these small\nimage perturbations. This highlights that state-of-the-art CNNs lack human level scene understanding\nand do not rely on the same causal features as humans for visual perception [4, 5, 11].\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fRegularization and implicit inductive biases in deep networks can positively affect robustness and\ngeneralization by constraining the parameter space and biasing the trained model to use better features.\nHowever, these biases are often rather nonspeci\ufb01c and networks often latch onto patterns that do\nnot generalize well outside the distribution of training data. Biological visual systems, however,\ncope with strongly varying conditions all the time. Based on recently reported overlap between\nthe sensory representations of task-trained CNNs and representations measured in primate brains\n[12, 13, 14, 15, 16], we thus hypothesized that biasing the representation of arti\ufb01cial networks towards\nbiological stimulus representations might positively affect their robustness.\nHere, we show that directly measuring the neural representation in animal visual cortices and biasing\nCNN models toward a more biological feature space can indeed lead to more robust models. To this\nend, we recorded the simultaneous responses of thousands of neurons to complex natural scenes\nin visual cortex of awake mice. In order to bias a CNN towards biological feature representations,\nwe modi\ufb01ed its objective function so that convolutional features are encouraged to establish the\nsame structure as neural activities. We found that by regularizing a ResNet [1] towards a biological\nneural representation, the trained models had higher classi\ufb01cation accuracy than baseline when input\nimages were corrupted by random noise or adversarial perturbations. Regularization towards random\nrepresentations or features from a pretrained VGG model was substantially less helpful.\n\n2 Neural representation similarity\n\nWe performed several 2-photon scans in primary visual cortex on multiple mice, with repeated scans\nper animal across different days. During the experiment, the head-\ufb01xed mice were able to run on a\ntreadmill while passively viewing natural image each presented for 500ms. In each experiment, we\nmeasured responses to 5100 different grayscale images sampled from the ImageNet dataset, 100 of\nwhich were repeated 10 times to give 6000 trials in total. Each image was downsampled by a factor\nof four to 64 \u00d7 36 pixels. We call the repeated images \u2018oracle images\u2019, because the mean neural\nresponses over these repeated trials serve as a high quality predictor (oracle) for validation trials. The\nmajor reason for choosing mice in our study is they allow for genetic tools for large scale recordings\n(\u223c8000 units simultaneously). While mice indeed do not have as sophisticated visual systems as\nprimates, vision is still one of their major sensory inputs. Grayscale images were used because mice\nare not sensitive to the colors relevant to human vision.\nWe begin by de\ufb01ning the similarity metric for neural responses, which we will then use to regularize\na CNN for image classi\ufb01cation. In a \ufb01rst step, the raw response \u03c1ai for each neuron a to stimulus i is\nscaled by its signal-to-noise ratio\n\nwa =\n\n\u03c3a\n\u03b7a\n\n,\n\n(1)\n\nwhich was estimated from responses to repeated stimuli, namely the oracle images. For a neuron a, the\na = Vari(Et[rait]) is the variance over stimuli i of the mean response over repeated\nsignal strength \u03c32\na = Ei[Vart(rait)].\ntrials t. The noise strength is the mean over stimuli of the variance over trials, \u03b72\nWe denote these scaled responses by rai = wa\u03c1ai. The scaled population response to stimulus i is\nthe vector ri. Scaling responses based on signal-to-noise ratio accounts for the reliability of each\nneuron by reducing the in\ufb02uence of noisy neurons. For example, if the responses of a neuron to the\nsame image are highly variable, we will ignore its contribution to the similarity metric by assigning a\nsmall weight to it, no matter how differently it responds to different images or how high its responses\nare in general.\nWe then shift and normalize these population responses, creating centered unit vectors ei = ri\u2212\u00afr\n(cid:107)ri\u2212\u00afr(cid:107)\nwhere \u00afr = Ei[ri] is the population response averaged over all stimuli. These unit vectors are then\nused to construct the similarity matrix, according to\n\nij = ei \u00b7 ej\nSdata\n\n(2)\n\nfor stimuli i and j.\n\n2.1 Stability across animals and days\n\nAveraging the responses to the repeated presentations of the oracle images allows us to reduce the\nin\ufb02uence of neural noise in the representation similarity metric de\ufb01ned in Eq. 2 and examine its\n\n2\n\n\fstability across scans (i.e. different selection of neurons). When calculating similarity between oracle\nimages, we can average the results of different trials to reduce noise. For given image i with T\nrepeats, we \ufb01rst treat those trials as if they are different images i1, . . . , iT , and calculate similarity\nagainst repeated trials of another oracle image j, (j1, . . . , jT ) in every combination. Oracle similarity\nis de\ufb01ned as the mean value of all trial similarity values\n\n(cid:104)\n\n(cid:105)\n\nSoracle\n\nij\n\n= Eti, tj\n\nSdata\niti jtj\n\n,\n\n(3)\n\n= 1 excluded when i = j.\n\nwith Sdata\nitit\nWe found that the neural representation similarity between images is stable across scans and across\nmice in primary visual cortex (Fig. 1A). When images (columns and rows) are ordered for better\nvisualization, there is a visible structure consistent across scans, revealing the clustering organization\nof these images. We further index the matrix for scan h as Soracle\u2212h\n, and compare the \ufb02uctuation\nacross scans\n\nij\n\n\u2206Sscan\n\nh,i,j = Soracle\u2212h\n\nij\n\n\u2212 Eh\n\n(cid:2)Soracle\u2212h\n\n(cid:3) ,\n\nij\n\n(4)\n\n(5)\n\nand the \ufb02uctuation across repeats\n\n\u2206Srepeat\nh,i,t1,t2\n\n= Sdata\u2212h\nit1 it2\n\n\u2212 Soracle\u2212h\n\nii\n\n.\n\nWe observer a much narrower distribution for \u2206Sscan than \u2206Srepeat (Fig. 1C), suggesting that the\nvariability due to the selection of neurons (scans) is much lower than the single trial variability to the\nsame image.\n\n2.2 Denoising neural responses with a predictive model\n\nij\n\n= \u02c6ei \u00b7 \u02c6ej .\n\nMost images in our experiments were only presented once to maximize the diversity of stimuli, so\nSoracle is not available for them while Sdata was too noisy for our purpose. To exploit the neural\nresponses for non-oracle images, we \ufb01rst train a predictive model to denoise data. The predictive\nmodel is consisted of a simple 3-layer CNN with skip connection [17, 18]. It takes images as inputs\nand predict neural responses by a linear readout at the last layer. In addition, behavioral data such as\nthe pupil position and size, as well as the running speed on the treadmill are also fed to the model to\naccount for the effect of non-visual variables.\nThe predicted response for neuron a to stimulus i is denoted as \u02c6\u03c1ai, which is trained to predict \u03c1ai\nwell [18]. The correlation between \u02c6\u03c1a and \u03c1a is denoted as va, indicating how well neuron a is\npredicted. The scaled model response is de\ufb01ned as \u02c6rai = wava \u02c6\u03c1ai with the sigal-to-noise weight wa\nfrom Eq. 1, and the population response is then denoted as \u02c6ri. The similarity matrix for scaled model\nresponses is calculated in the same way as Eq. 2,\nSmodel\n\n(6)\nSimilarity matrices for the same set of oracle images are shown in Fig. 1B, each from a model trained\nfor the corresponding scan. The similarity for measured neural responses, Soracle, are also present\nin the model response similarities, but the structure is more prominent for the model responses. A\nscatter plot of data and model similarities, Soracle\n(Fig. 1D), shows a high correlation\nr = 0.73, but the model similarities have a wider range. In the same plot we also showed the\ncorrelation between Soracle and the corresponding trial similarity values Sdata from which they are\nestimated, and found Smodel to be much less noisy than Sdata.\nThe use of model neuron responses as a proxy for the real neurons has three major bene\ufb01ts. First, the\noutputs are deterministic, eliminating the random noise component. Second, the predictive model was\nheavily regularized during training, so these deterministic responses are more likely to re\ufb02ect reliable\nvisual features. Third, the model\u2019s shifter and modulator circuit [17] accounted for the irrelevant\nnonvisual eye and body movements, and could thereby extract more of the purely visual-driven\nresponses.\nWith the help of a predictive model, we can obtain cleaner responses for the 5000 non-oracle images\neven though they are only measured once. We used the similarity matrices averaged over 8 scans\nas the regularization target. Two examples of the model neural similarity for the 100 oracle images\nare shown in Fig. 2. It is worth clarifying that we don\u2019t use this 100\u00d7100 matrix in our main result\nthough, but only the 5000\u00d75000 matrix from non-oracle trials. Oracle trials are used for evaluating\npredictive models, assigning neuron-speci\ufb01c weights and demonstrations (Fig. 1 and 2) only.\n\nversus Smodel\n\nij\n\nij\n\n3\n\n\fFigure 1: Representation similarity in neural data and predictive models. Image order is preserved\nacross all matrices. (A) Similarity matrices Soracle (Eq. 3) from real neural responses to 100 oracle\nimages. The structure of similarity matrices is stable across scans on different animals and days. (B)\nSimilarity matrices Smodel (Eq. 6) from predictive models. The backbone of representation similarity\nis preserved and enhanced. (C) Variability over scans is smaller than that over repeats. (D) Similarity\nvalues estimated from single trials are noisy (Sdata vs. Soracle, r = 0.39), but similarity calculated\nfrom predictive models correlate with those from neural data well (Smodel vs. Soracle, r = 0.73). In\naddition, Smodel spans a wider range than Soracle, makes it a better training target in practice.\n\n3 Neural regularization by joint training\n\nTo regularize a standard machine learning model with the representation similarity matrix obtained\nfrom neural data [19], we jointly train the model with a similarity loss in addition to its original\ntask-de\ufb01ned loss (Fig. 3, also see [20] and [21] for related approaches based on fMRI or other deep\nneuronal networks, respectively). The full loss function contains two terms, de\ufb01ned as\n\nL = Ltask + \u03b1Lsimilarity .\n\n(7)\n\nThe \ufb01rst term is a conventional loss used to de\ufb01ne the performance on the task, such as classi\ufb01cation\nor 1-shot learning. In this section, we implement grayscale CIFAR10 classi\ufb01cation, hence we use\na cross-entropy loss. The second term is the penalty that favors brain-like representations, with a\ncoef\ufb01cient \u03b1 determining regularization strength.\nFor any pair of images that were shown to the mice, we already have their representational similarity\nfrom models predicting neural data (Eq. 6). Since we are now comparing similarity for two models, a\n\n4\n\nfrom real neuron (averaged)-0.200.20.40.60.820457-5-920457-7-1020457-8-1220892-3-1420505-10-1420505-11-1620505-12-2920505-14-33from model neuron (averaged)-0.500.5120892-3-1420505-14-3320457-5-920457-7-1020457-8-1220505-10-1420505-11-1620505-12-29representation similarity for oracle imagesimage id1100-0.500.5r=0.3858\u0394\u0394-0.200.20.40.60.8-0.200.20.40.600.51-0.200.20.40.6r=0.7276ABCD\fFigure 2: Examples of similar and dissimilar image pairs. From the similarity matrix of model neural\nresponses, examples of low similarity value (left pair) and high similarity value (right pair) are shown.\n\nneural predictive model and a task model based on a convolutional neural network, we denote the\nformer by Sneural\nWe de\ufb01ne the similarity loss for image i and image j as\n\n. We want SCNN to approximate Sneural well.\n\nand the latter by SCNN\n\nij\n\nij\n\nLsimilarity =(cid:2)arctanh(cid:0)SCNN\n\n(cid:1) \u2212 arctanh(cid:0)Sneural\n\n(cid:1)(cid:3)2\n\n.\n\nij\n\nij\n\n(8)\nThe arctanh is used to remap the similarities from the interval [\u22121, 1] to (\u2212\u221e,\u221e). It is analogous\nto the Fisher transform which uses the same arctanh function to compute con\ufb01dence intervals for\ncorrelation coef\ufb01cients, by reparameterizing the correlations to follow nearly normal distributions.\nWhen similarity values are not too close to \u22121 or 1, the loss is close to the sample based centered\nkernel alignment (CKA) index [22, 23, 24].\nIntuitively, SCNN is the cosine similarity of convolutional features that image i and j activate. Though\nV1 responses are thought to encode low-level features, there\u2019s no principled way to determine a priori\nwhich single model layer corresponds to V1. Thus we \ufb02exibly combine feature similarities from\na selection of layers instead of assigning to a speci\ufb01c one. Speci\ufb01cally, we calculate similarity for\nK uniformly located convolutional layers, and average the results through a trainable weight. The\nweights are the outputs of a softmax function, therefore guaranteed to be positive and sum to one.\nMathematically speaking, for each of the K layers we compute the cosine similarity values as\n\n(cid:16)\ni \u2212 \u00aff (k)(cid:17) \u00b7(cid:16)\n(cid:13)(cid:13)(cid:13)f (k)\ni \u2212 \u00aff (k)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)(cid:13)f (k)\n\nj \u2212 \u00aff (k)(cid:17)\nj \u2212 \u00aff (k)(cid:13)(cid:13)(cid:13) ,\n\nf (k)\n\nf (k)\n\nSCNN\u2212k\n\nij\n\n=\n\n(9)\n\n(cid:104)\n\n(cid:105)\n\ni\n\nf (k)\n\nSCNN\n\nij =\n\nwhere f (k)\nEi\ni\nlayers\n\nis the concatenated convolutional feature vector for image i at layer k, and \u00aff (k) =\nis its mean over images. The \ufb01nal model similarity is a combination from all selected\n\nwhere \u03b3k is a trainable probability with(cid:80)\n\n(cid:88)\n(10)\nk \u03b3k = 1, \u03b3k \u2265 0. This means that the objective function\ncan choose which layer to match the similarity, but it needs to match at least one in total as enforced\nby the softmax that determines \u03b3k. In principle all convolutional layers can be included, but we\nonly used 5 in our simulations (layer 1, 5, 9, 13, 17 in ResNet18). The preliminary analysis shows\nafter training one layer will dominate in Eq. 10, and it is typically layer 5, the last layer of the \ufb01rst\nResBlock group. More details are included in the supplementary materials.\nIn each step of training, we \ufb01rst process a batch of CIFAR images to calculate classi\ufb01cation loss\nLclassi\ufb01cation, and subsequently process a batch of image pairs sampled from the stimuli we used\n\n\u03b3kSCNN\u2212k\n\nij\n\nk\n\n,\n\n5\n\nmodel neuron representation similarity-0.500.51\fFigure 3: Joint training schematic. We trained a ResNet18 model to both classify CIFAR10 images\nand predict neural similarity of ImageNet images used in our scan. The network takes either one\nimage or a pair of images as inputs, with a same convolutional core. If the input is one image with\nthe right size, the model outputs class prediction with an additional fully connected layer. If the input\nis a pair of images, the model \ufb01rst calculate the convolutional features for both, and calculate the\nsimilarity for a few selected layers (Eq. 10). Similarity predictions from different layers are summed\nup by a trainable normalized weight to produce a \ufb01nal prediction, which is trained to match neural\nsimilarity (Eq. 6). Two losses are summed with a coef\ufb01cient \u03b1 as the regularization strength.\n\nin experiments, calculating the similarity loss Lsimilarity with respect to the pre-computed Sneural\nmatrix. The gradient of the full loss can affect the CNN kernel weights through both loss terms.\n\n4 Results\n\n4.1 Robustness against random noise\n\nThe similarity loss plays the role of a regularizer, and it biases the original CNN towards a more\nbrain-like representation. We observed that the CNN model becomes more robust to random noise\nwhen neural regularization is used. Compared to a ResNet18 [1] trained without any regularization\n(\u2018None\u2019 in Fig. 4A), the same architecture equipped with the neural regularizer (\u2018Neural (model)\u2019 in\nFig. 4) had substantially better performance on noisy input images (\u223c50% v.s. \u223c20% at the highest\nnoise level). In other words, models whose features are more neural are less vulnerable to random\nnoise in inputs. To strengthen this conclusion, we also regularized the model with shuf\ufb02ed Sneural\nmatrix (\u2018Shuf\ufb02e\u2019 in Fig. 4) or the feature similarity matrix of the conv3-1 layer in a VGG19 model\npretrained on ImageNet (\u2018VGG\u2019 in Fig. 4). This VGG layer has been reported to be most similar to\nanimal V1 [16]. Both regularizers improve the model robustness to some degree but neither as much\nas using the neural regularizer.\nFinally, we also regularized the model with a similarity matrix from the actual data directly (\u2018Neural\n(data)\u2019 in Fig. 4), using Sdata (Eq. 2) instead of Smodel (Eq. 6). We did not observe the same boost in\nrobustness. We think that this is caused by the high variability of the neural responses, highlighting\nthe need for a well trained predictive model. In addition, if we see the matrix in \u2018Shuf\ufb02e\u2019 control as\nthe feature similarity of a poorly trained predictive model, the difference between \u2018Neural (model)\u2019\nand \u2018Shuf\ufb02e\u2019 again shows the importance of having a well trained one. Only with a strong predictive\nsystem identi\ufb01cation model as a denoiser were we able to reveal the underlying representational\nstructure hidden in the noisy neural data.\nWe observed the same results when training ResNet34 models on grayscale CIFAR100 datasets\n(Fig. 4B). In addition, we also tested how different regularization strength will affect the model perfor-\nmance, and observed a continuous increase of model robustness when we tuned up the regularization.\nMore details are included in the supplementary materials.\n\n6\n\nclassi(cid:31)cationsimilarity predictionloss = lossclassi(cid:31)cation+ losssimilarityResNet18\fFigure 4: Performance robustness to Gaussian noise. (A) We tested CIFAR10 classi\ufb01cation perfor-\nmance under different levels of Gaussian noises on input images (examples below the plot) for our\njointly trained ResNet model, and compared with models with no regularization and some other\nregularization. Compared to the vanilla network with no regularization (\u2018None\u2019), all regularized\nmodel have higher classi\ufb01cation accuracy when discernible noise is added. In particular, the model\nregularized with model neural similarity outperforms others on noisy images, only with a small\nsacri\ufb01ce on clean image performance. The error bars here are standard error of mean (SEM), with 5\nrandom seeds used for each regularizer. The reduced improvement from \u2018Neural (data)\u2019 emphasizes\nthe need for a good predictive model for denoising, so that the actual neural representation structure\ncan be exploited. (B) Results for ResNet34 on grayscale CIFAR100 dataset are shown for \u2018None\u2019,\n\u2018Neural (model)\u2019 and \u2018Shuf\ufb02e\u2019.\n\nAll models are trained by stochastic gradient descent for 40 epochs with batch size 64. Learning\nrate starts at 0.1 and decays by 0.3 every 4 epochs, but resets to 0.1 after the 20th epoch. Mean\nclassi\ufb01cation accuracy for CIFAR10/100 test set over 5 random seeds is reported in Fig. 4. In our\ncurrent setting, the same number of images are passed in the classi\ufb01cation pathway and neural\npathway, hence the time cost approximately doubles comparing to normal training. It takes about 4.5\nhours on a single TITAN RTX GPU to train one model. We used PyTorch [25] for model training.\n\n4.2 Robustness against adversarial attack\n\nWe are also interested in whether neural regularization provides robustness to adversarial attacks.\nSince adversarial examples and their innocent counterparts elicit the same percept by de\ufb01nition, it is\nhighly possible that their measured neural representations are also close to each other. Hence a model\nwith neural representation will be more invariant to adversarial noise. We evaluated model robustness\nfollowing a recently published guideline [26] and using the well-tested attack implementations\nprovided by Foolbox [27].\nOur evaluation metric follows [28]. In a nutshell, we strive to \ufb01nd adversarial perturbations (i.e.\nperturbations that \ufb02ip the label to any but the ground-truth class) with the minimum norm (either L2\nor L\u221e) for each of 1000 test samples. We then compute the median perturbation distance across all\nsamples as the \ufb01nal robustness score (higher is better).\nBesides the current state-of-the-art attacks on L2 [26] and L\u221e [29], we also deployed a recently\ndeveloped gradient-based version of the decision-based boundary attack [30], which surpasses [26]\nin terms of query ef\ufb01ciency and the size of the minimal adversarial perturbations. In short, [30] starts\nfrom a natural input sample that is classi\ufb01ed as different from the original image (for which we aim to\ngenerate an adversarial example). The algorithm then performs a line search between the two images\n\n7\n\n00.020.040.060.08Gaussian noise level20406080accuracy (%)NoneNeural (model)Neural (data)ShuffleVGGRegularization00.020.040.060.08010203040506070NoneNeural (model)ShuffleGaussian noise levelResNet18 on grayscale CIFAR10ResNet34 on grayscale CIFAR100AB\fto \ufb01nd the decision boundary of the model. The gradients w.r.t. the difference between the two\ntop-most logits allow us to estimate the local geometry of the decision boundary. Using this geometry\nwe can compute the optimal adversarial perturbation that (a) takes us exactly to the boundary (in\ncase we are slightly shifted away from it), (b) stays within the valid pixel bounds, (c) minimizes the\ndistances to the original image, and (d) is not too far from the current perturbation (to make sure we\nstay within the region for which the linear approximation of the boundary is valid). Therefore, our\ngradient-based version of the decision boundary attack provides a most stringent test for adversarial\nrobustness of our neural network machine learning models regularized with neural data.\nTo ensure that we evaluate and compare all models fairly, we perform an extensive hyperparameter\nsearch and always select the optimal combination. Since our gradient-based boundary attack proved\nmore effective than [26] on all models tested here, we only deployed the gradient-based boundary\nattack for L2, and used Projected Gradient Descent (PGD) [29] for L\u221e in our \ufb01nal evaluation. For\nour gradient-based boundary attack, we tested step sizes of {0.0003, 0.001, 0.003, 0.01, 0.03, 0.1,\n0.03}, and for PGD we tested step sizes of {10\u22126, 10\u22125, 10\u22124, 10\u22123, 10\u22122, 10\u22121, 1} with iterations\nof {10, 30, 50, 100, 200}.\nFig. 5 shows that regularizing models with neural representational similarity improves model robust-\nness against adversarial attacks. The model with the smallest adversarial perturbations (most fragility)\nis the vanilla model trained without any regularization (median perturbation of 0.0025 (L\u221e) and\n0.09 (L2)). Regularizing with random similarity matrix (median perturbation of 0.003 (L\u221e) and 0.11\n(L2)) or similarity of VGG features (median perturbation of 0.0028 (L\u221e) and 0.11 (L2)) increases\nrobustness. The strongest increase in robustness, in both metrics, is provided by the regularization\nwith the brain\u2019s representations learned from neural data (median perturbation of 0.0034 (L\u221e) and\n0.13 (L2)).\nWe additionally did a more thorough experiment with a few more type of attacks, and looked at L0\nand L1 metrics. More details are included in the supplementary materials.\n\nFigure 5: Adversarial robustness of classi\ufb01er networks according to the L2 and L\u221e norms. With\nmore optimization queries for the attack, the minimal perturbation shrinks. Regularization improves\nadversarial robustness, with neural regularization providing the best defense throughout the attacker\u2019s\noptimization. \u2018Neural (data)\u2019 regularizer is not tested.\n\nWe observed increased robustness across several metrics, which is quite remarkable given that current\ndefense methods, in particular adversarial training, tend to over\ufb01t strongly on the metrics on which\nthey are trained on [28] and are often less robust on other Lp metrics than undefended models.\n\n5 Conclusion and discussion\n\nNeuroscience has often provided inspiration to machine learning, but it lacks methods to directly\ntranslate neurophysiological recordings into an improvement of arti\ufb01cial neural networks. Here,\nwe have shown that regularization with neural data proves to be a promising tool to create an\ninductive bias towards more robust inference. In particular, the mouse brain has evolved an image\nrepresentation that is capable of performing dif\ufb01cult machine learning tasks, but is more robust than\nconventional models. Speci\ufb01cally, we demonstrated that when these measured representations are\nincorporated into a general machine learning model by matching representational similarity, they\nenable more robust machine learning algorithms (robustness to random noise and adversarial attacks).\n\n8\n\n0.10.20.30.40.0030.004NoneNeural (model)ShuffleVGGRegularization\fCritically, our predictive computational model provided a better assessment of representation than\nraw neural data, because it disentangles non-visual features from visual ones, and transforms the\nisolated visual features into a reliable, de-noised version of neural responses. This model-based\nrepresentation proved useful as a regularization target for machine learning models. We conjecture\nthat by regularizing machine learning models further to match representational similarities with higher\norder visual areas beyond V1 will further enhance the robustness and generalization performance\noutside the training set. These brain-like representations may help machine learning algorithms\nultimately reach human-like performance.\nThere are at least two ways to regularize CNN models to favor neural representations. One is to learn\nthe similarity for any pair of images, like our approach here. The other is to jointly train a linear\nreadout from intermediate layers of CNN to predict neural responses directly. However, we argue\nthat the former is a tighter constraint since a wide range of af\ufb01ne transformations in the CNN could\nbe compensated by the linear readout, producing identical predictions for the neural responses while\nsubstantially altering the underlying representational similarity in the CNN. For this reason, we chose\nto regularize our machine learning models to match the representational similarity.\nThough the improvement of adversarial robustness by neural regularization is substantial and sig-\nni\ufb01cant, unsurprisingly, the current state-of-the-art in terms of robustness on L\u221e [29] remains\nsubstantially more robust than our neurally regularized but otherwise undefended model (0.029 vs\n0.0034). That said, [29] employs an expensive adversarial training procedure that\u2014in contrast to our\nmethod\u2014speci\ufb01cally aims to optimize robustness against L\u221e perturbations. As a side effect, [29]\nperforms signi\ufb01cantly worse on metrics it has not been trained on, such as L2 or L0 [28] while our\nmethod does not over\ufb01t on one speci\ufb01c metric. Combining the neural regularization with adversarial\ntraining procedure [29] could potentially lead to even stronger defenses.\nThe neural regularization is not designed to improve model robustness, but rather to bias any model\nto have neural features. We expect to see other bene\ufb01ts with such inductive bias, such as improved\ngeneralization in domain transfer, lower sample complexity in few-shot learning, and so on. While\nmore systematic analysis is continuing, preliminary results indeed have shown improvement by neural\nregularization in those aspects as well.\nTo bias CNN features towards a more brain-like representation, we matched the pairwise cosine\nsimilarity for a given set of inputs in this study. But this is just one approach of manifold matching in\na more general sense [31]. We will explore other metrics or higher-order dependencies in the future.\nWhile our results indeed show the bene\ufb01t of adopting more brain-like representation in visual\nprocessing, it is however unclear which aspects of neural representation make it work. We think that\nit is the most important question and we need to understand the principle behind it. There are two\napproaches that we are currently working on. The \ufb01rst is to directly compare the regularized models\nand the vanilla ones by investigating the features they use. We will look into the tuning property\nof model units by \ufb01nding the input patterns that maximally excites these units, and examine how\nneural regularization makes a difference. The second is to identify which neurons are useful for a\nmore robust representation. We can either \ufb01nd the subset of neurons that are most important in the\nsimilarity metric and look for their common properties. Or we can propose some criterion to select a\nparticular set of neurons and check whether using those neurons alone can obtain the same robustness\ngain. If we manage to understand why neural regularization works, we\u2019ll be able to design or train\nmachine learning models just with the underlying principles, without actually performing large-scale\nneural recordings.\nA docker image containing all codes and trained models is prepared (zheli18/neural-reg:neurips19),\nwith a jupyter lab as entrypoint.\n\nAcknowledgements\n\nThis work is supported by the Intelligence Advanced Research Projects Activity (IARPA) via De-\npartment of Interior/Interior Business Center (DoI/IBC) contract number D16PC00003. The U.S.\nGovernment is authorized to reproduce and distribute reprints for Governmental purposes notwith-\nstanding any copyright annotation thereon. Disclaimer: The views and conclusions contained herein\nare those of the authors and should not be interpreted as necessarily representing the of\ufb01cial policies\nor endorsements, either expressed or implied, of IARPA, DoI/IBC, or the U.S. Government. FS\n\n9\n\n\fis supported by the Institutional Strategy of the University of T\u00fcbingen (Deutsche Forschungsge-\nmeinschaft, ZUK 63) and the Carl-Zeiss-Stiftung. FS and MB acknowledges the support from the\nGerman Federal Ministry of Education and Research (BMBF) through the T\u00fcbingen AI Center\n(FKZ: 01IS18039A) and the DFG Cluster of Excellence \u201cMachine Learning \u2013 New Perspectives for\nScience\u201d, EXC 2064/1, project number 390727645.\n\nReferences\n[1] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In The\nIEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770\u2013778, 2016.\n\n[2] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. CoRR, abs/1709.01507,\n\n2017.\n\n[3] Robert Geirhos, Carlos R. M. Temme, Jonas Rauber, Heiko H. Sch\u00fctt, Matthias Bethge, and\nFelix A. Wichmann. Generalisation in humans and deep neural networks.\nIn S. Bengio,\nH. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in\nNeural Information Processing Systems 31, pages 7538\u20137550. Curran Associates, Inc., 2018.\n\n[4] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. Imagenet-\ntrained cnns are biased towards texture; increasing shape bias improves accuracy and robustness.\nICLR, pages 1\u201321, 2019.\n\n[5] Wieland Brendel and Matthias Bethge. Approximating CNNs with bag-of-local-features models\nworks surprisingly well on imagenet. In International Conference on Learning Representations,\n2019.\n\n[6] Fabio Anselmi, Lorenzo Rosasco, and Tomaso Poggio. On invariance and selectivity in\nrepresentation learning. Information and Inference: A Journal of the IMA, 5(2):134\u2013158, 05\n2016.\n\n[7] Tomaso A. Poggio and Fabio Anselmi. Visual Cortex and Deep Networks: Learning Invariant\n\nRepresentations. The MIT Press, 1st edition, 2016.\n\n[8] Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. Building\n\nmachines that learn and think like people. Behavioral and Brain Sciences, 40:e253, 2017.\n\n[9] Andrea Tacchetti, Leyla Isik, and Tomaso A. Poggio. Invariant recognition shapes neural\n\nrepresentations of visual input. Annual Review of Vision Science, 4(1):403\u2013422, 2018.\n\n[10] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Good-\n\nfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv e-prints, 2013.\n\n[11] Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and\nAleksander Madry. Adversarial examples are not bugs, they are features. arXiv e-prints, May\n2019.\n\n[12] Daniel L. K. Yamins, Ha Hong, Charles F. Cadieu, Ethan A. Solomon, Darren Seibert, and\nJames J. DiCarlo. Performance-optimized hierarchical models predict neural responses in higher\nvisual cortex. Proceedings of the National Academy of Sciences, 111(23):8619\u20138624, 2014.\n\n[13] Seyed-Mahdi Khaligh-Razavi and Nikolaus Kriegeskorte. Deep supervised, but not unsuper-\nvised, models may explain it cortical representation. PLOS Computational Biology, 10(11):1\u201329,\n2014.\n\n[14] Umut G\u00fc\u00e7l\u00fc and Marcel A. J. van Gerven. Deep neural networks reveal a gradient in the\ncomplexity of neural representations across the ventral stream. Journal of Neuroscience,\n35(27):10005\u201310014, 2015.\n\n[15] Daniel L. K. Yamins and James J. DiCarlo. Using goal-driven deep learning models to under-\n\nstand sensory cortex. Nature Neuroscience, 19:356, 2016.\n\n10\n\n\f[16] Santiago A. Cadena, George H. Den\ufb01eld, Edgar Y. Walker, Leon A. Gatys, Andreas S. Tolias,\nMatthias Bethge, and Alexander S. Ecker. Deep convolutional models improve predictions of\nmacaque v1 responses to natural images. PLOS Computational Biology, 15(4):e1006897, 2019.\n\n[17] Fabian Sinz, Alexander S Ecker, Paul Fahey, Edgar Walker, Erick Cobos, Emmanouil\nFroudarakis, Dimitri Yatsenko, Zachary Pitkow, Jacob Reimer, and Andreas Tolias. Stim-\nulus domain transfer in recurrent models for large scale cortical population prediction on video.\nIn S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors,\nAdvances in Neural Information Processing Systems 31, pages 7199\u20137210. Curran Associates,\nInc., 2018.\n\n[18] Edgar Y. Walker, Fabian H. Sinz, Emmanouil Froudarakis, Paul G. Fahey, Taliah Muhammad,\nAlexander S. Ecker, Erick Cobos, Jacob Reimer, Xaq Pitkow, and Andreas S. Tolias. Inception\nin visual cortex: in vivo-silico loops reveal most exciting images. bioRxiv, 2018.\n\n[19] Nikolaus Kriegeskorte, Marieke Mur, and Peter Bandettini. Representational similarity analysis\n- connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2:4,\n2008.\n\n[20] Nathaniel Blanchard, Jeffery Kinnison, Brand on RichardWebster, Pouya Bashivan, and Walter J.\nScheirer. A neurobiological evaluation metric for neural network model search. arXiv e-prints,\npage arXiv:1805.10726, 2018.\n\n[21] Patrick McClure and Nikolaus Kriegeskorte. Representational distance learning for deep neural\n\nnetworks. Frontiers in Computational Neuroscience, 10:131, 2016.\n\n[22] Nello Cristianini, John Shawe-Taylor, Andr\u00e9 Elisseeff, and Jaz S. Kandola. On kernel-target\nalignment. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural\nInformation Processing Systems 14, pages 367\u2013373. MIT Press, 2002.\n\n[23] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. Algorithms for learning kernels\n\nbased on centered alignment. J. Mach. Learn. Res., 13(1):795\u2013828, 2012.\n\n[24] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural\n\nnetwork representations revisited. arXiv e-prints, page arXiv:1905.00414, 2019.\n\n[25] Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito,\nZeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in\nPyTorch. In NIPS Autodiff Workshop, 2017.\n\n[26] N. Carlini and D. Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE\n\nSymposium on Security and Privacy (SP), pages 39\u201357, 2017.\n\n[27] Jonas Rauber, Wieland Brendel, and Matthias Bethge. Foolbox: A python toolbox to benchmark\n\nthe robustness of machine learning models. arXiv e-prints, page arXiv:1707.04131, 2017.\n\n[28] Lukas Schott, Jonas Rauber, Matthias Bethge, and Wieland Brendel. Towards the \ufb01rst ad-\nversarially robust neural network model on MNIST. arXiv e-prints, page arXiv:1805.09190,\n2018.\n\n[29] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu.\nTowards deep learning models resistant to adversarial attacks. In International Conference on\nLearning Representations, 2018.\n\n[30] Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reli-\nable attacks against black-box machine learning models. arXiv e-prints, page arXiv:1712.04248,\n2017.\n\n[31] A. Talwalkar, S. Kumar, and H. Rowley. Large-scale manifold learning.\n\nConference on Computer Vision and Pattern Recognition, pages 1\u20138, June 2008.\n\nIn 2008 IEEE\n\n11\n\n\f", "award": [], "sourceid": 5059, "authors": [{"given_name": "Zhe", "family_name": "Li", "institution": "Baylor College of Medicine"}, {"given_name": "Wieland", "family_name": "Brendel", "institution": "AG Bethge, University of T\u00fcbingen"}, {"given_name": "Edgar", "family_name": "Walker", "institution": "Baylor College of Medicine"}, {"given_name": "Erick", "family_name": "Cobos", "institution": "Baylor College of Medicine"}, {"given_name": "Taliah", "family_name": "Muhammad", "institution": "Baylor College of Medicine"}, {"given_name": "Jacob", "family_name": "Reimer", "institution": "Baylor College of Medicine"}, {"given_name": "Matthias", "family_name": "Bethge", "institution": "University of T\u00fcbingen"}, {"given_name": "Fabian", "family_name": "Sinz", "institution": "University T\u00fcbingen"}, {"given_name": "Zachary", "family_name": "Pitkow", "institution": "BCM/Rice"}, {"given_name": "Andreas", "family_name": "Tolias", "institution": "Baylor College of Medicine"}]}