The topic of the paper is very relevant to the NeurIPS community, given the increased interest in understanding self-supervised learning. Reviewers have appreciated the direction the paper takes for this, ie, to study invariances learned by self-supervised learning methods, comparing them with supervised representations. There were some concerns about the interpretations of the emprical results which have been addressed in the author response. This paper takes the first and important step towards understanding the invariances in self-supervised representations and their implications on downstream tasks, and would be of interest to the NeurIPS community. I recommend acceptance.