The paper investigates various data augmentation techniques in the context of RL, and shows that they lead to improved performance. The method is simple and can be applied to different RL algorithms. One may argue that the algorithmic contribution is not majorly novel, but the simplicity of the method and the improvement in the performance, as well as the unanimous favourable opinion of the reviewers, would be good enough justification to recommend acceptance of this work. I encourage the authors to consider the comments of the reviewers in revising their paper. I would also like to ask the authors to increase the number of runs/seeds in some of their experiments from 3 or 4 to a larger number (10+). Currently, the difference between methods may not be statistically significant in some cases. I believe having a spotlight is suitable for this work, as many RL practitioner can benefit from knowing about the positive effect of data augmentation in RL.