NeurIPS 2020

Can I Trust My Fairness Metric? Assessing Fairness with Unlabeled Data and Bayesian Inference

Meta Review

This paper focuses on the problem of leveraging unlabelled data to generate better estimates of fairness metrics given limited labelled data. All three reviewers agree that the manuscript makes a valuable contribution and is conceptually and mathematically sound. The significance of the contribution (an auditor tool only, instead of an auditor plus a mitigation tool) is however at the low side.