{"title": "A simple neural network module for relational reasoning", "book": "Advances in Neural Information Processing Systems", "page_first": 4967, "page_last": 4976, "abstract": "Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn. In this paper we describe how to use Relation Networks (RNs) as a simple plug-and-play module to solve problems that fundamentally hinge on relational reasoning. We tested RN-augmented networks on three tasks: visual question answering using a challenging dataset called CLEVR, on which we achieve state-of-the-art, super-human performance; text-based question answering using the bAbI suite of tasks; and complex reasoning about dynamical physical systems. Then, using a curated dataset called Sort-of-CLEVR we show that powerful convolutional networks do not have a general capacity to solve relational questions, but can gain this capacity when augmented with RNs. Thus, by simply augmenting convolutions, LSTMs, and MLPs with RNs, we can remove computational burden from network components that are not well-suited to handle relational reasoning, reduce overall network complexity, and gain a general ability to reason about the relations between entities and their properties.", "full_text": "A simple neural network module\n\nfor relational reasoning\n\nAdam Santoro\u2217\n\nDavid Raposo\u2217\n\nDavid G.T. Barrett\n\nadamsantoro@google.com\n\ndraposo@google.com\n\nbarrettdavid@google.com\n\nMateusz Malinowski\nmateuszm@google.com\n\nRazvan Pascanu\nrazp@google.com\n\nPeter Battaglia\n\npeterbattaglia@google.com\n\nTimothy Lillicrap\n\nDeepMind\n\nLondon, United Kingdom\ncountzero@google.com\n\nAbstract\n\nRelational reasoning is a central component of generally intelligent behavior, but\nhas proven dif\ufb01cult for neural networks to learn. In this paper we describe how to\nuse Relation Networks (RNs) as a simple plug-and-play module to solve problems\nthat fundamentally hinge on relational reasoning. We tested RN-augmented net-\nworks on three tasks: visual question answering using a challenging dataset called\nCLEVR, on which we achieve state-of-the-art, super-human performance; text-\nbased question answering using the bAbI suite of tasks; and complex reasoning\nabout dynamic physical systems. Then, using a curated dataset called Sort-of-\nCLEVR we show that powerful convolutional networks do not have a general\ncapacity to solve relational questions, but can gain this capacity when augmented\nwith RNs. Thus, by simply augmenting convolutions, LSTMs, and MLPs with\nRNs, we can remove computational burden from network components that are\nnot well-suited to handle relational reasoning, reduce overall network complexity,\nand gain a general ability to reason about the relations between entities and their\nproperties.\n\n1\n\nIntroduction\n\nThe ability to reason about the relations between entities and their properties is central to generally\nintelligent behavior (Figure 1) [10, 7]. Consider a child proposing a race between the two trees in the\npark that are furthest apart: the pairwise distances between every tree in the park must be inferred and\ncompared to know where to run. Or, consider a reader piecing together evidence to predict the culprit\nin a murder-mystery novel: each clue must be considered in its broader context to build a plausible\nnarrative and solve the mystery.\nSymbolic approaches to arti\ufb01cial intelligence are inherently relational [16, 5]. Practitioners de\ufb01ne\nthe relations between symbols using the language of logic and mathematics, and then reason about\nthese relations using a multitude of powerful methods, including deduction, arithmetic, and algebra.\nBut symbolic approaches suffer from the symbol grounding problem and are not robust to small\n\n\u2217Equal contribution.\n\n31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA.\n\n\fFigure 1: An illustrative example from the CLEVR dataset of relational reasoning. An image\ncontaining four objects is shown alongside non-relational and relational questions. The relational\nquestion requires explicit reasoning about the relations between the four objects in the image, whereas\nthe non-relational question requires reasoning about the attributes of a particular object.\n\ntask and input variations [5]. Other approaches, such as those based on statistical learning, build\nrepresentations from raw data and often generalize across diverse and noisy conditions [12]. However,\na number of these approaches, such as deep learning, often struggle in data-poor problems where the\nunderlying structure is characterized by sparse but complex relations [3, 11]. Our results corroborate\nthese claims, and further demonstrate that seemingly simple relational inferences are remarkably\ndif\ufb01cult for powerful neural network architectures such as convolutional neural networks (CNNs)\nand multi-layer perceptrons (MLPs).\nHere, we explore \u201cRelation Networks\u201d (RN) as a general solution to relational reasoning in neural\nnetworks. RNs are architectures whose computations focus explicitly on relational reasoning [18].\nAlthough several other models supporting relation-centric computation have been proposed, such\nas Graph Neural Neworks, Gated Graph Sequence Neural Netoworks, and Interaction Networks,\n[20, 13, 2], RNs are simpler, more exclusively focused on general relation reasoning, and easier to\nintegrate within broader architectures. Moreover, RNs require minimal oversight to construct their\ninput, and can be applied successfully to tasks even when provided with relatively unstructured inputs\ncoming from CNNs and LSTMs. We applied an RN-augmented architecture to CLEVR [7], a recent\nvisual question answering (QA) dataset on which state-of-the-art approaches have struggled due to the\ndemand for rich relational reasoning. Our networks vastly outperformed the best generally-applicable\nvisual QA architectures, and achieve state-of-the-art, super-human performance. RNs also solve\nCLEVR from state descriptions, highlighting their versatility in regards to the form of their input. We\nalso applied an RN-based architecture to the bAbI text-based QA suite [22] and solved 18/20 of the\nsubtasks. Finally, we trained an RN to make challenging relational inferences about complex physical\nsystems and motion capture data. The success of RNs across this set of substantially dissimilar task\ndomains is testament to the general utility of RNs for solving problems that require relation reasoning.\n\n2 Relation Networks\n\nAn RN is a neural network module with a structure primed for relational reasoning. The design\nphilosophy behind RNs is to constrain the functional form of a neural network so that it captures the\ncore common properties of relational reasoning. In other words, the capacity to compute relations is\nbaked into the RN architecture without needing to be learned, just as the capacity to reason about\nspatial, translation invariant properties is built-in to CNNs, and the capacity to reason about sequential\ndependencies is built into recurrent neural networks.\nIn its simplest form the RN is a composite function:\n\nRN(O) = f\u03c6\n\ng\u03b8(oi, oj)\n\n(1)\n\n\uf8f6\uf8f8 ,\n\n\uf8eb\uf8ed(cid:88)\n\ni,j\n\n2\n\nWhat is the size of the brown sphere?Non-relational question:Original Image:Relational question:Are there any rubber things that have the same size as the yellow metallic cylinder?\fwhere the input is a set of \u201cobjects\u201d O = {o1, o2, ..., on}, oi \u2208 Rm is the ith object, and f\u03c6 and g\u03b8\nare functions with parameters \u03c6 and \u03b8, respectively. For our purposes, f\u03c6 and g\u03b8 are MLPs, and the\nparameters are learnable synaptic weights, making RNs end-to-end differentiable. We call the output\nof g\u03b8 a \u201crelation\u201d; therefore, the role of g\u03b8 is to infer the ways in which two objects are related, or if\nthey are even related at all.\nRNs have three notable strengths: they learn to infer relations, they are data ef\ufb01cient, and they operate\non a set of objects \u2013 a particularly general and versatile input format \u2013 in a manner that is order\ninvariant.\n\nRNs learn to infer relations The functional form in Equation 1 dictates that an RN should consider\nthe potential relations between all object pairs. This implies that an RN is not necessarily privy to\nwhich object relations actually exist, nor to the actual meaning of any particular relation. Thus, RNs\nmust learn to infer the existence and implications of object relations.\nIn graph theory parlance, the input can be thought of as a complete and directed graph whose nodes\nare objects and whose edges denote the object pairs whose relations should be considered. Although\nwe focus on this \u201call-to-all\u201d version of the RN throughout this paper, this RN de\ufb01nition can be\nadjusted to consider only some object pairs. Similar to Interaction Networks [2], to which RNs are\nrelated, RNs can take as input a list of only those pairs that should be considered, if this information\nis available. This information could be explicit in the input data, or could perhaps be extracted by\nsome upstream mechanism.\n\nRNs are data ef\ufb01cient RNs use a single function g\u03b8 to compute each relation. This can be thought\nof as a single function operating on a batch of object pairs, where each member of the batch is a\nparticular object-object pair from the same object set. This mode of operation encourages greater\ngeneralization for computing relations, since g\u03b8 is encouraged not to over-\ufb01t to the features of any\nparticular object pair. Consider how an MLP would learn the same function. An MLP would receive\nall objects from the object set simultaneously as its input. It must then learn and embed n2 (where n\nis the number of objects) identical functions within its weight parameters to account for all possible\nobject pairings. This quickly becomes intractable as the number of objects grows. Therefore, the cost\nof learning a relation function n2 times using a single feedforward pass per sample, as in an MLP, is\nreplaced by the cost of n2 feedforward passes per object set (i.e., for each possible object pair in the\nset) and learning a relation function just once, as in an RN.\n\nRNs operate on a set of objects The summation in Equation 1 ensures that the RN is invariant to\nthe order of objects in its input, respecting the property that sets are order invariant. Although we\nused summation, other commutative operators \u2013 such as max, and average pooling \u2013 can be used\ninstead.\n\n3 Tasks\n\nWe applied RN-augmented networks to a variety of tasks that hinge on relational reasoning. To\ndemonstrate the versatility of these networks we chose tasks from a number of different domains,\nincluding visual QA, text-based QA, and dynamic physical systems.\n\n3.1 CLEVR\n\nIn visual QA a model must learn to answer questions about an image (Figure 1). This is a challenging\nproblem domain because it requires high-level scene understanding [1, 14]. Architectures must\nperform complex relational reasoning \u2013 spatial and otherwise \u2013 over the features in the visual inputs,\nlanguage inputs, and their conjunction. However, the majority of visual QA datasets require reasoning\nin the absence of fully speci\ufb01ed word vocabularies, and perhaps more perniciously, a vast and\ncomplicated knowledge of the world that is not available in the training data. They also contain\nambiguities and exhibit strong linguistic biases that allow a model to learn answering strategies that\nexploit those biases, without reasoning about the visual input [1, 15, 19].\nTo control for these issues, and to distill the core challenges of visual QA, the CLEVR visual QA\ndataset was developed [7]. CLEVR contains images of 3D-rendered objects, such as spheres and\ncylinders (Figure 2). Each image is associated with a number of questions that fall into different\n\n3\n\n\fcategories. For example, query attribute questions may ask \u201cWhat is the color of the sphere?\u201d,\nwhile compare attribute questions may ask \u201cIs the cube the same material as the cylinder?\u201d.\nFor our purposes, an important feature of CLEVR is that many questions are explicitly relational in\nnature. Remarkably, powerful QA architectures [24] are unable to solve CLEVR, presumably because\nthey cannot handle core relational aspects of the task. For example, as reported in the original paper a\nmodel comprised of ResNet-101 image embeddings with LSTM question processing and augmented\nwith stacked attention modules vastly outperformed other models at an overall performance of 68.5%\n(compared to 52.3% for the next best, and 92.6% human performance) [7]. However, for compare\nattribute and count questions (i.e., questions heavily involving relations across objects), the\nmodel performed little better than the simplest baseline, which answered questions solely based on\nthe probability of answers in the training set for a given question category (Q-type baseline).\nWe used two versions of the CLEVR dataset: (i) the pixel version, in which images were represented\nin standard 2D pixel form. (ii) a state description version, in which images were explicitly represented\nby state description matrices containing factored object descriptions. Each row in the matrix contained\nthe features of a single object \u2013 3D coordinates (x, y, z); color (r, g, b); shape (cube, cylinder, etc.);\nmaterial (rubber, metal, etc.); size (small, large, etc.). When we trained our models, we used either\nthe pixel version or the state description version, depending on the experiment, but not both together.\n\n3.2 Sort-of-CLEVR\n\nTo explore our hypothesis that the RN architecture is better suited to general relational reasoning as\ncompared to more standard neural architectures, we constructed a dataset similar to CLEVR that we\ncall \u201cSort-of-CLEVR\u201d2. This dataset separates relational and non-relational questions.\nSort-of-CLEVR consists of images of 2D colored shapes along with questions and answers about the\nimages. Each image has a total of 6 objects, where each object is a randomly chosen shape (square\nor circle). We used 6 colors (red, blue, green, orange, yellow, gray) to unambiguously identify each\nobject. Questions are hard-coded as \ufb01xed-length binary strings to reduce the dif\ufb01culty involved\nwith natural language question-word processing, and thereby remove any confounding dif\ufb01culty\nwith language parsing. For each image we generated 10 relational questions and 10 non-relational\nquestions. Examples of relational questions are: \u201cWhat is the shape of the object that is farthest from\nthe gray object?\u201d; and \u201cHow many objects have the same shape as the green object?\u201d. Examples of\nnon-relational questions are: \u201cWhat is the shape of the gray object?\u201d; and \u201cIs the blue object on the\ntop or bottom of the scene?\u201d. The dataset is also visually simple, reducing complexities involved in\nimage processing.\n\n3.3 bAbI\n\nbAbI is a pure text-based QA dataset [22]. There are 20 tasks, each corresponding to a particular\ntype of reasoning, such as deduction, induction, or counting. Each question is associated with a set\nof supporting facts. For example, the facts \u201cSandra picked up the football\u201d and \u201cSandra went to\nthe of\ufb01ce\u201d support the question \u201cWhere is the football?\u201d (answer: \u201cof\ufb01ce\u201d). A model succeeds on\na task if its performance surpasses 95%. Many memory-augmented neural networks have reported\nimpressive results on bAbI. When training jointly on all tasks using 10K examples per task, Memory\nNetworks pass 14/20, DNC 16/20, Sparse DNC 19/20, and EntNet 16/20 (the authors of EntNets\nreport state-of-the-art at 20/20; however, unlike previously reported results this was not done with\njoint training on all tasks, where they instead achieve 16/20) [23, 4, 17, 6].\n\n3.4 Dynamic physical systems\n\nWe developed a dataset of simulated physical mass-spring systems using the MuJoCo physics engine\n[21]. Each scene contained 10 colored balls moving on a table-top surface. Some of the balls moved\nindependently, free to collide with other balls and the barrier walls. Other randomly selected ball\npairs were connected by invisible springs or a rigid constraint. These connections prevented the balls\nfrom moving independently, due to the force imposed through the connections. Input data consisted\nof state descriptions matrices, where each ball was represented as a row in a matrix with features\n\n2The \u201cSort-of-CLEVR\u201d dataset will be made publicly available online\n\n4\n\n\frepresenting the rgb color values of each object and their spatial coordinates (x and y) across 16\nsequential time steps.\nThe introduction of random links between balls created an evolving physical system with a variable\nnumber \u201csystems\u201d of connected balls (where \u201csystems\u201d refers to connected graphs with balls as\nnodes and connections between balls as edges). We de\ufb01ned two separate tasks: 1) infer the existence\nor absence of connections between balls when only observing their color and coordinate positions\nacross multiple sequential frames, and 2) count the number of systems on the table-top, again when\nonly observing each ball\u2019s color and coordinate position across multiple sequential frames.\nBoth of these tasks involve reasoning about the relative positions and velocities of the balls to infer\nwhether they are moving independently, or whether their movement is somehow dependent on the\nmovement of other balls through invisible connections. For example, if the distance between two\nballs remains similar across frames, then it can be inferred that there is a connection between them.\nThe \ufb01rst task makes these inferences explicit, while the second task demands that this reasoning\noccur implicitly, which is much more dif\ufb01cult. For further information on all tasks, including videos\nof the dynamic systems, see the supplementary information.\n\n4 Models\n\nIn their simplest form RNs operate on objects, and hence do not explicitly operate on images or\nnatural language. A central contribution of this work is to demonstrate the \ufb02exibility with which\nrelatively unstructured inputs, such as CNN or LSTM embeddings, can be considered as a set of\nobjects for an RN. As we describe below, we require minimal oversight in factorizing the RN\u2019s input\ninto a set of objects.\n\nFigure 2: Visual QA architecture. Questions are processed with an LSTM to produce a question\nembedding, and images are processed with a CNN to produce a set of objects for the RN. Objects\n(three examples illustrated here in yellow, red, and blue) are constructed using feature-map vectors\nfrom the convolved image. The RN considers relations across all pairs of objects, conditioned on the\nquestion embedding, and integrates all these relations to answer the question.\n\nDealing with pixels We used a CNN to parse pixel inputs into a set of objects. The CNN took\nimages of size 128 \u00d7 128 and convolved them through four convolutional layers to k feature maps of\nsize d \u00d7 d, where k is the number of kernels in the \ufb01nal convolutional layer. We remained agnostic as\nto what particular image features should constitute an object. So, after convolving the image, each of\nthe d2 k-dimensional cells in the d \u00d7 d feature maps was tagged with a coordinate (from the range\n(\u22121, 1) for each of the x- and y-coordinates)3 indicating its relative spatial position, and was treated\nas an object for the RN (see Figure 2). This means that an \u201cobject\u201d could comprise the background, a\nparticular physical object, a texture, conjunctions of physical objects, etc., which affords the model\ngreat \ufb02exibility in the learning process.\n\n3We also experimented without this tagging, and achieved performance of 88% on the validation set.\n\n5\n\nsmall+-MLP............isspherewhatsizeFinal CNN feature mapsRNLSTMobject-MLPConv.What size is the cylinder that is left of the brown metal thing that is left of the big sphere?*Object pair with questionElement-wise sum\fg\u03b8 could condition its processing on the question: a = f\u03c6((cid:80)\n\nConditioning RNs with question embeddings The existence and meaning of an object-object\nrelation should be question dependent. For example, if a question asks about a large sphere, then the\nrelations between small cubes are probably irrelevant. So, we modi\ufb01ed the RN architecture such that\ni,j g\u03b8(oi, oj, q)). To get the question\nembedding q, we used the \ufb01nal state of an LSTM that processed question words. Question words\nwere assigned unique integers, which were then used to index a learnable lookup table that provided\nembeddings to the LSTM. At each time-step, the LSTM received a single word embedding as input,\naccording to the syntax of the English-encoded question.\n\nDealing with state descriptions We can provide state descriptions directly into the RN, since state\ndescriptions are pre-factored object representations. Question processing can proceed as before:\nquestions pass through an LSTM using a learnable lookup embedding for individual words, and the\n\ufb01nal state of the LSTM is concatenated to each object-pair.\n\nDealing with natural language For the bAbI suite of tasks the natural language inputs must be\ntransformed into a set of objects. This is a distinctly different requirement from visual QA, where\nobjects were de\ufb01ned as spatially distinct regions in convolved feature maps. So, we \ufb01rst took the\n20 sentences in the support set that were immediately prior to the probe question. Then, we tagged\nthese sentences with labels indicating their relative position in the support set, and processed each\nsentence word-by-word with an LSTM (with the same LSTM acting on each sentence independently).\nWe note that this setup invokes minimal prior knowledge, in that we delineate objects as sentences,\nwhereas previous bAbI models processed all word tokens from all support sentences sequentially.\nIt\u2019s unclear how much of an advantage this prior knowledge provides, since period punctuation also\nunambiguously delineates sentences for the token-by-token processing models. The \ufb01nal state of the\nsentence-processing-LSTM is considered to be an object. Similar to visual QA, a separate LSTM\nproduced a question embedding, which was appened to each object pair as input to the RN. Our\nmodel was trained on the joint version of bAbI (all 20 tasks simultaneously), using the full dataset of\n10K examples per task.\n\nModel con\ufb01guration details For the CLEVR-from-pixels task we used: 4 convolutional layers\neach with 24 kernels, ReLU non-linearities, and batch normalization; 128 unit LSTM for question\nprocessing; 32 unit word-lookup embeddings; four-layer MLP consisting of 256 units per layer with\nReLU non-linearities for g\u03b8; and a three-layer MLP consisting of 256, 256 (with 50% dropout), and\n29 units with ReLU non-linearities for f\u03c6. The \ufb01nal layer was a linear layer that produced logits for a\nsoftmax over the answer vocabulary. The softmax output was optimized with a cross-entropy loss\nfunction using the Adam optimizer with a learning rate of 2.5e\u22124. We used size 64 mini-batches\nand distributed training with 10 workers synchronously updating a central parameter server. The\ncon\ufb01gurations for the other tasks are similar, and can be found in the supplementary information.\nWe\u2019d like to emphasize the simplicity of our overall model architecture compared to the visual QA\narchitectures used on CLEVR thus far, which use ResNet or VGG embeddings, sometimes with\n\ufb01ne-tuning, very large LSTMs for language encoding, and further processing modules, such as\nstacked or iterative attention, or large fully connected layers (upwards of 4000 units, often) [7].\n\n5 Results\n\n5.1 CLEVR from pixels\n\nOur model achieved state-of-the-art performance on CLEVR at 95.5%, exceeding the best model\ntrained only on the pixel images and questions at the time of the dataset\u2019s publication by 27%, and\nsurpassing human performance in the task (see Table 1 and Figure 3).\nThese results \u2013 in particular, those obtained in the compare attribute and count categories \u2013 are\na testament to the ability of our model to do relational reasoning. In fact, it is in these categories that\nstate-of-the-art models struggle most. Furthermore, the relative simplicity of the network components\nused in our model suggests that the dif\ufb01culty of the CLEVR task lies in its relational reasoning\ndemands, not on the language or the visual processing.\nMany CLEVR questions involve computing and comparing more than one relation; for example,\nconsider the question: \u201cThere is a big thing on the right side of the big rubber cylinder that is behind\n\n6\n\n\fModel\n\nOverall\n\nCount\n\nExist\n\nCompare\nNumbers\n\nQuery\nAttribute\n\nCompare\nAttribute\n\nHuman\nQ-type baseline\nLSTM\nCNN+LSTM\nCNN+LSTM+SA\nCNN+LSTM+SA*\nCNN+LSTM+RN\n\n92.6\n41.8\n46.8\n52.3\n68.5\n76.6\n95.5\n\n86.7\n34.6\n41.7\n43.7\n52.2\n64.4\n90.1\n\n96.6\n50.2\n61.1\n65.2\n71.1\n82.7\n97.8\n\n86.5\n51.0\n69.8\n67.1\n73.5\n77.4\n93.6\n\n95.0\n36.0\n36.8\n49.3\n85.3\n82.6\n97.9\n\n96.0\n51.3\n51.8\n53.0\n52.3\n75.4\n97.1\n\n* Our implementation, with optimized hyperparameters and trained end-to-end using the same\nCNN as in our RN model. We also tagged coordinates, which did not improve performance.\n\nTable 1: Results on CLEVR from pixels. Performances of our model (RN) and previously reported\nmodels [8], measured as accuracy on the test set and broken down by question category.\n\nthe large cylinder to the right of the tiny yellow rubber thing; What is its shape?\u201d, which has three\nspatial relations (\u201cright side\u201d, \u201cbehind\u201d, \u201cright of\u201d). On such questions, our model achieves 93%\nperformance, indicating that the model can handle complex relational reasoning.\n\nResults using privileged training information A more recent study reports overall performance\nof 96.9% on CLEVR, but uses additional supervisory signals on the functional programs used to\ngenerate the CLEVR questions [8]. It is not possible for us to directly compare this to our work since\nwe do not use these additional supervision signals. Nonetheless, our approach greatly outperforms\na version of their model that was not trained with these extra signals, and even a version of their\nmodel trained using 9K ground-truth programs. Thus, RNs can achieve very competitive, and even\nsuper-human results under much weaker and more natural assumptions, and even in situations when\nfunctional programs are unavailable.\n\nFigure 3: Results on CLEVR from pixels. The RN-augmented model outperformed all other models\nand exhibited super-human performance overall. In particular, it solved \u201ccompare attribute\u201d questions,\nwhich trouble all other models because they heavily depend on relational reasoning.\n\n5.2 CLEVR from state descriptions\n\nTo demonstrate that the RN is robust to the form of its input, we trained our model on the state\ndescription matrix version of the CLEVR dataset. The model achieved an accuracy of 96.4%. This\nresult demonstrates the generality of the RN module, showing its capacity to learn and reason\nabout object relations while being agnostic to the kind of inputs it receives \u2013 i.e., to the particular\nrepresentation of the object features to which it has access. Therefore, RNs are not necessarily\n\n7\n\nequallessthanmorethanexistcountoverallquerysizequeryshapequerymaterialquerycolorcomparesizecompareshapecomparematerialcomparecolorHumanLSTMCNN+LSTMCNN+LSTM+SACNN+LSTM+RNQ-type baselineAccuracyAccuracy0.00.250.50.751.00.00.250.50.751.0compare attributequery attributecompare numbers\frestricted to visual problems, and can thus be applied in very different contexts, and to different tasks\nthat require relational reasoning.\n\n5.3 Sort-of-CLEVR from pixels\n\nThe results so far led us to hypothesize that the dif\ufb01culty in solving CLEVR lies in its heavy emphasis\non relational reasoning, contrary to previous claims that the dif\ufb01culty lies in question parsing [9].\nHowever, the questions in the CLEVR dataset are not categorized based on the degree to which they\nmay be relational, making it hard to assess our hypothesis. Therefore, we use the Sort-of-CLEVR\ndataset which we explicitly designed to seperate out relational and non-relational questions (see\nSection 3.2).\nWe \ufb01nd that a CNN augmented with an RN achieves an accuracy above 94% for both relational and\nnon-relational questions. However, a CNN augmented with an MLP only reached this performance\non the non-relational questions, plateauing at 63% on the relational questions. This strongly indicates\nthat models lacking a dedicated relational reasoning component struggle, or may even be completely\nincapable of solving tasks that require very simple relational reasoning. Augmenting these models\nwith a relational module, like the RN, is suf\ufb01cient to overcome this hurdle.\nA simple \u201cclosest-to\u201d or \u201cfurthest-from\u201d relation is particularly revealing of a CNN+MLP\u2019s lack\nof general reasoning capabilities (52.3% success). For these relations a model must gauge the\ndistances between each object, and then compare each of these distances. Moreover, depending on\nthe images, the relevant distance could be quite small in magnitude, or quite large, further increasing\nthe combinatoric dif\ufb01culty of this task.\n\n5.4 bAbI\n\nOur model succeeded on 18/20 tasks. Notably, it succeeded on the basic induction task (2.1%\ntotal error), which proved dif\ufb01cult for the Sparse DNC (54%), DNC (55.1%), and EntNet (52.1%).\nAlso, our model did not catastrophically fail in any of the tasks: for the 2 tasks that it failed (the\n\u201ctwo supporting facts\u201d, and \u201cthree supporting facts\u201d tasks), it missed the 95% threshold by 3.1%\nand 11.5%, respectively. We also note that the model we evaluated was chosen based on overall\nperformance on a withheld validation set, using a single seed. That is, we did not run multiple replicas\nwith the best hyperparameter settings (as was done in other models, such as the Sparse DNC, which\ndemonstrated performance \ufb02uctuations with a standard deviation of more than \u00b13 tasks passed for\nthe best choice of hyperparameters).\n\n5.5 Dynamic physical systems\n\nFinally, we trained our model on two tasks requiring reasoning about the dynamics of balls moving\nalong a surface. In the connection inference task, our model correctly classi\ufb01ed all the connections in\n93% of the sample scenes in the test set. In the counting task, the RN achieved similar performance,\nreporting the correct number of connected systems for 95% of the test scene samples. In comparison,\nan MLP with comparable number of parameters was unable to perform better than chance for both\ntasks. Moreover, using this task to learn to infer relations results in transfer to unseen motion capture\ndata, where RNs predict the connections between body joints of a walking human (see Supplementary\nMaterial for experimental details and example videos).\n\n6 Discussion and Conclusions\n\nRNs are powerful, versatile, and simple neural network modules with the capacity for relational\nreasoning. The performance of RN-augmented networks on CLEVR is especially notable; they\nsigni\ufb01cantly improve upon current general purpose, state-of-the-art models (upwards of 25%),\nindicating that previous architectures lacked a fundamental, general capacity to reason about relations.\nMoreover, these results unveil an important distinction between the often confounded notions of\nprocessing and reasoning. Powerful visual QA architectures contain components, such as ResNets,\nwhich are highly capable visual processors capable of detecting complicated textures and forms.\nHowever, as demonstrated by CLEVR, they lack an ability to reason about the features they detect.\n\n8\n\n\fRNs can easily exploit foreknowledge of the relations that should be computed for a particular task.\nIndeed, especially in circumstances with strong computational constraints, bounding the otherwise\nquadratic complexity of the number of relations could be advantageous. Attentional mechanisms\ncould reduce the number of objects fed as input to the RN, and hence reduce the number of relations\nthat need to be considered. Or, using an additional down-sampling convolutional or pooling layer\ncould further reduce the number of objects provided as input to the RN; indeed, max-pooling to 4 \u00d7 4\nfeature maps reduces the total number of objects, and hence computed relations, and results in 87%\nperformance on the validation set.\nRNs have a \ufb02exible input format: a set of objects. Our results show that, strikingly, the set of objects\ndoes not need to be cleverly pre-factored. RNs learn to deal with \u201cobject\u201d representations provided\nby CNNs and LSTMs, presumably by in\ufb02uencing the content and form of the object representations\nvia the gradients they propagate.\nIn future work it would be interesting to apply RNs to relational reasoning across highly abstract\nentities (for example, decisions in hierarchical reinforcement learning tasks). Relation reasoning is a\ncentral component of generally intelligent behavior, and so, we expect the RN to be a simple-to-use,\nuseful and widely used neural module.\n\nAcknowledgments\n\nWe would like to thank Murray Shanahan, Ari Morcos, Scott Reed, Daan Wierstra, and many others\non the DeepMind team, for critical feedback and discussions.\n\n9\n\n\fReferences\n[1] Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick,\n\nand Devi Parikh. Vqa: Visual question answering. In ICCV, 2015.\n\n[2] Peter Battaglia, Razvan Pascanu, Matthew Lai, Danilo Jimenez Rezende, et al. Interaction networks for\n\nlearning about objects, relations and physics. In NIPS, 2016.\n\n[3] Marta Garnelo, Kai Arulkumaran, and Murray Shanahan. Towards deep symbolic reinforcement learning.\n\narXiv:1609.05518, 2016.\n\n[4] Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwi\u00b4nska,\nSergio G\u00f3mez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, et al. Hybrid computing\nusing a neural network with dynamic external memory. Nature, 2016.\n\n[5] Stevan Harnad. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3):335\u2013346,\n\n1990.\n\n[6] Mikael Henaff, Jason Weston, Arthur Szlam, Antoine Bordes, and Yann LeCun. Tracking the world state\n\nwith recurrent entity networks. In ICLR, 2017.\n\n[7] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross\nGirshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In\nCVPR, 2017.\n\n[8] Justin Johnson, Bharath Hariharan, Laurens van der Maaten, Judy Hoffman, Li Fei-Fei, C Lawrence\nZitnick, and Ross Girshick. Inferring and executing programs for visual reasoning. arXiv:1705.03633,\n2017.\n\n[9] Kushal Ka\ufb02e and Christopher Kanan.\n\narXiv:1703.09684, 2017.\n\nAn analysis of visual question answering algorithms.\n\n[10] Charles Kemp and Joshua B Tenenbaum. The discovery of structural form. Proceedings of the National\n\nAcademy of Sciences, 105(31):10687\u201310692, 2008.\n\n[11] Brenden M Lake, Tomer D Ullman, Joshua B Tenenbaum, and Samuel J Gershman. Building machines\n\nthat learn and think like people. arXiv:1604.00289, 2016.\n\n[12] Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436\u2013444, 2015.\n[13] Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph sequence neural networks.\n\nICLR, 2016.\n\n[14] Mateusz Malinowski and Mario Fritz. A multi-world approach to question answering about real-world\n\nscenes based on uncertain input. In NIPS, 2014.\n\n[15] Mateusz Malinowski, Marcus Rohrbach, and Mario Fritz. Ask your neurons: A deep learning approach to\n\nvisual question answering. arXiv:1605.02697, 2016.\n\n[16] Allen Newell. Physical symbol systems. Cognitive science, 4(2):135\u2013183, 1980.\n[17] Jack Rae, Jonathan J Hunt, Ivo Danihelka, Timothy Harley, Andrew W Senior, Gregory Wayne, Alex\nGraves, and Tim Lillicrap. Scaling memory-augmented neural networks with sparse reads and writes. In\nNIPS, 2016.\n\n[18] David Raposo, Adam Santoro, David Barrett, Razvan Pascanu, Timothy Lillicrap, and Peter Battaglia.\n\nDiscovering objects and their relations from entangled scene representations. arXiv:1702.05068, 2017.\n\n[19] Mengye Ren, Ryan Kiros, and Richard Zemel. Image question answering: A visual semantic embedding\n\nmodel and a new dataset. In NIPS, 2015.\n\n[20] Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The\n\ngraph neural network model. IEEE Transactions on Neural Networks, 2009.\n\n[21] Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control. In\n\nIROS, 2012.\n\n[22] Jason Weston, Antoine Bordes, Sumit Chopra, and Tomas Mikolov. Towards ai-complete question\n\nanswering: A set of prerequisite toy tasks. arXiv:1502.05698, 2015.\n\n[23] Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. In ICLR, 2015.\n[24] Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alex Smola. Stacked attention networks for image\n\nquestion answering. In CVPR, 2016.\n\n10\n\n\f", "award": [], "sourceid": 2565, "authors": [{"given_name": "Adam", "family_name": "Santoro", "institution": "DeepMind"}, {"given_name": "David", "family_name": "Raposo", "institution": "DeepMind"}, {"given_name": "David", "family_name": "Barrett", "institution": "DeepMind"}, {"given_name": "Mateusz", "family_name": "Malinowski", "institution": "DeepMind"}, {"given_name": "Razvan", "family_name": "Pascanu", "institution": "Google DeepMind"}, {"given_name": "Peter", "family_name": "Battaglia", "institution": "DeepMind"}, {"given_name": "Timothy", "family_name": "Lillicrap", "institution": "Google DeepMind"}]}