NeurIPS 2020

A Fair Classifier Using Kernel Density Estimation

Meta Review

The paper proposes a simple but rather practical approach to estimate statistical fairness notions without relying on a proxy, in contrast to several prior work. The proposed approach relies on Kernel Density Estimation (KDE), which allows to compute the gradient of the fairness notion with respect to the model parameters in close form, easing the learning procedure of a fair classifier. As a result, he proposed approach leads to a better fairness accuracy trade-off than competing methods in several datasets. In particular, the experiments show that the proposed approach outperforms prior work relying on fairness proxies, and leads more stable results that approaches that rely on adversarial training top trade-off fairness and accuracy. In fact, the empirical results are comparable to the ones provided by Agarwal et al. (2018), whose solution provide theoretical guarantees but comes at a high computational cost. Although there exists extensive literature on solving the fair classification problem, the empirical results show the efficacy of KDE in this context. Moreover, this work may also trigger future work, where the KDE may be used to solve the highly non-convex original constraint optimization problem (rather than the regularized problem), navigate the Pareto frontier between fairness and accuracy, or solve the problem of fair decision making, e.g., under selective labels where classification approaches are not optimal anymore. The reviewers agree on the simplicity and potential usefulness of the proposed approach--e.g., R1 mentions that "The approach is straightforward, applicable to many model classes and could be a practical way of encoding fairness concerns in real AI systems." The authors properly responded in the rebuttal to the major points raised by the reviewers, being the main concern that remains the need of additional experiments. In the rebuttal, the authors do already mention their extension and synthetic experiments to the multi-class settings and multiple sensitive attributes, as well as their plan to conduct an ablation study and more detail comparison with [44]. Based on such detailed experiment description, I believe that this is an easy fix for the camera-ready, and I encourage the authors to thoroughly incorporate all the reviewers' feedback in the revised version of their paper, as it will significantly improve the potential impact of the paper.