The paper proposes an approach for incorporating knowledge about symmetries or equivariances into neural network policies by providing a general purpose method for constructing network layers based on knowledge of the relevant transformations. The reviews are generally positive: Identifying effective ways of incorporating prior knowledge of this type into neural networks is an important research challenge that is of interest to the community. The proposed approach for constructing network layers seems novel, although there is some prior work that explores ways of exploiting such knowledge in particular application domains, or via alternative means such as data augmentation. (Much of this is cited in the paper.) An important caveat of the submission, remarked upon by all reviewers is the experimental evaluation. It is currently limited to simple scenarios with perfect symmetries which provide limited evidence of the utility of the approach in more complex / less idealized scenarios. Furthermore, important baselines such as data augmentation approaches were added only in the rebuttal. Although the paper is generally clearly written the explanation of how invariant multi-layer networks are constructed could be approved. The reviewers discussed the paper extensively. All reviewers think that the idea will be useful to the community and has the potential to spark a discussion. Disagreement remained, however, whether the experimental evaluation is sufficient in the current form. On balance all reviewers feel positive about the paper. The meta reviewer would strongly encourage the authors to incorporate the data augmentation baselines as well as the results for Breakout (and / or some other non-idealized domain) into the final version. The authors may want to include the following paper in their related work section: https://arxiv.org/pdf/1909.10707.pdf