Overall the reviewers agreed that the strengths of the paper outweighed its weaknesses and were in favor of acceptance. In preparing the paper for publication the authors should seek to address R3's outstanding concern regarding the results on the COMPAS data. Given that this data has come to be used as a benchmark, not being able to provide a clear explanation of why the approach does not work on COMPAS (why race is not computationally identifiable) is of critical importance.