{"title": "Unlocking Fairness: a Trade-off Revisited", "book": "Advances in Neural Information Processing Systems", "page_first": 8783, "page_last": 8792, "abstract": "The prevailing wisdom is that a model's fairness and its accuracy\n are in tension with one another. However, there is a pernicious\n {\\em modeling-evaluating dualism} bedeviling fair machine learning\n in which phenomena such as label bias are appropriately acknowledged\n as a source of unfairness when designing fair models,\n only to be tacitly abandoned when evaluating them. We investigate\n fairness and accuracy, but this time under a variety of controlled\n conditions in which we vary the amount and type of bias. We find,\n under reasonable assumptions, that the tension between fairness and\n accuracy is illusive, and vanishes as soon as we account for these\n phenomena during evaluation. Moreover, our results are consistent\n with an opposing conclusion: fairness and accuracy are sometimes in\n accord. This raises the question, {\\em might there be a way to\n harness fairness to improve accuracy after all?} Since most\n notions of fairness are with respect to the model's predictions and\n not the ground truth labels, this provides an opportunity to see if\n we can improve accuracy by harnessing appropriate notions of\n fairness over large quantities of {\\em unlabeled} data with\n techniques like posterior regularization and generalized\n expectation. Indeed, we find that semi-supervision not only\n improves fairness, but also accuracy and has advantages over\n existing in-processing methods that succumb to selection bias on the\n training set.", "full_text": "Unlocking Fairness: a Trade-off Revisited\n\nMichael Wick, Swetasudha Panda, Jean-Baptiste Tristan\n\n{michael.wick,swetasudha.panda,jean.baptiste.tristan}@oracle.com\n\nOracle Labs, Burlington, MA.\n\nAbstract\n\nThe prevailing wisdom is that a model\u2019s fairness and its accuracy are in tension\nwith one another. However, there is a pernicious modeling-evaluating dualism\nbedeviling fair machine learning in which phenomena such as label bias are ap-\npropriately acknowledged as a source of unfairness when designing fair models,\nonly to be tacitly abandoned when evaluating them. We investigate fairness and\naccuracy, but this time under a variety of controlled conditions in which we vary the\namount and type of bias. We \ufb01nd, under reasonable assumptions, that the tension\nbetween fairness and accuracy is illusive, and vanishes as soon as we account for\nthese phenomena during evaluation. Moreover, our results are consistent with an\nopposing conclusion: fairness and accuracy are sometimes in accord. This raises\nthe question, might there be a way to harness fairness to improve accuracy after\nall? Since many notions of fairness are with respect to the model\u2019s predictions\nand not the ground truth labels, this provides an opportunity to see if we can im-\nprove accuracy by harnessing appropriate notions of fairness over large quantities\nof unlabeled data with techniques like posterior regularization and generalized\nexpectation. We \ufb01nd that semi-supervision improves both accuracy and fairness\nwhile imparting bene\ufb01cial properties of the unlabeled data on the classi\ufb01er.\n\n1\n\nIntroduction\n\nTorrents of ink have been spilled characterizing the relationship between a classi\ufb01er\u2019s \u201cfairness\u201d\nand its accuracy [11, 7, 3, 8, 20, 4, 14, 2, 13, 17], where fairness refers to a concrete mathematical\nembodiment of some rule provided by an external party such as a government and which must be\nimposed on a learning algorithm. The consensus, countenanced by both empirical and analytical\nstudies, is that the relationship is a trade-off: satisfying the supplied fairness constraints is achieved\nonly at the expense of accuracy. On the one hand, these \ufb01ndings are intuitive: if we think of fairness\nas constraints limiting the set of possible classi\ufb01cation assignments to those that are collectively fair,\nthen clearly accuracy suffers because in general, optimization over the subset always lower bounds\noptimization over the original set. As put in another paper \u201cdemanding fairness of models always\ncome at a cost of reduced accuracy\u201d [2].1\nOn the other hand, the belief in a simple assumption immediately calls these \ufb01ndings into question.\nIn particular, it requires no stretch of credulity to imagine that various personal attributes (e.g., race,\ngender, religion; sometimes termed \u201cprotected attributes\u201d) have no bearing on a person\u2019s intelligence,\ncapability, potential, quali\ufb01cations, etc., and consequently no bearing on ground truth classi\ufb01cation\nlabels \u2014 such as job quali\ufb01cation status \u2014 that might be functions of these qualities.2 It then follows\nthat enforcing fairness across these attributes should on average increase accuracy. The reason is clear.\nIf our classi\ufb01er produces different label distributions depending on the values of these dimensions,\nthen we know, under the foregoing assumption, that at least one of these distributions must be wrong,\nand thus there is an opportunity to improve accuracy. An opportunity to which we later return.\n\n1Our emphasis.\n2This assumption is consistent with the \u201cwe\u2019re all equal\u201d worldview [9]\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fBut \ufb01rst we must understand what accounts for this antinomy. Two possible explanations involve the\nphenomena of label bias and selection bias. Label bias occurs when the process that produces the\nlabels (e.g., a manual annotation process or a decision making process such as hiring) are in\ufb02uenced\nby factors that are not particularly germane to the determination of the label value, and thus might\ndiffer from the ideal labels, whatever they should have been. Accuracy measured against any such\nbiased labels should be considered carefully with a grain of salt. Selection bias occurs when selecting\na subsample of the data in such a way that happens to introduce unexpected correlations, say, between\na protected attribute and the target label. Training data, which is usually derived via selection from a\nlarger set of unlabeled data and subsequently frozen in time, is especially prone to this problem.\nIf pressed to couch the above discussion in a formal framework such as probably approximately\ncorrect (PAC) learning, we would say that we have a data distribution D and labeling function f,\neither of which could be biased. For example, due to selection bias we might have a \ufb02awed data\ndistribution D0 and due to label bias we might have a \ufb02awed labeling function f0. This leads to four\nregimes: the data distribution is biased (D0) or not (D) and the labeling function is biased (f0) or not\n(f). Many theoretical works in fair machine learning consider the regime in which neither is biased,\nand many empirical works\u2014due in part to the unavailability of an unbiased f\u2014draw conclusions\nassuming the regime in which neither is biased. But many forms of unfairness arise exactly because\none or both of these are biased: hence the dualism in fair machine learning. In this work, we assume\nthat some of the unfairness might arrise because we are actually in one of the other three regimes.\nIn this paper we account for both label and selection bias in our evaluations and show that when taken\ninto consideration, that certain de\ufb01nitions of fairness and accuracy are not always in tension. Since\nwe do not have access to the unbiased, unobserved ground truth labels in practice, we instead simulate\ndatasets in tightly controlled ways such that, for example, it exposes the actual unbiased labels for\nevaluation. Encouraged by theoretical results on semi-supervised PAC learning that state that these\ntechniques will be successful exactly when there is compatibility between some semi-supervised\nsignal and the data distribution [1] and the success of GE [16, 10], we also introduce and study a\nsemi-supervised method that exploits fairness constraints expressed over large quantities of unlabeled\ndata to build better classi\ufb01ers. Indeed, we \ufb01nd that as fairness improves, so does accuracy. Moreover,\nwe \ufb01nd that like other fairness methods, the semi-supervised approach can successfully overcome\nlabel bias; but unlike other fairness methods, it can also overcome selection bias on the training set.\n\n2 Related work\n\nSomehow, the idea that fairness and accuracy are not always in tension is both obvious and incon-\nspicuous (but nevertheless of practical signi\ufb01cance). The idea appears obvious because we assume\nthe unobserved unbiased ground-truth to be fair, and then limit our hypotheses to the fair region of\nthe space, and then claim that fairness improves accuracy. At this level of generality, it even appears\nto beg the question, but note that not all fair hypotheses are accurate since in the case we consider\na perfectly random classi\ufb01er is also fair. Moreover, the noise on the observed biased labels with\nwhich we train the classi\ufb01er is diametrically opposed to the unobserved label. Thus even under our\nassumptions, it is not a foregone conclusion that improving fairness improves accuracy. Rather, our\nassumption merely leaves open the possibility for this to happen. The \ufb01nding is inconspicuous in\nthe sense that, as mentioned earlier, there is a preponderance of work investigating this trade-off\nyet label bias appears to have gone unnoticed: very few papers (e.g., [8, 20]) mention the fact that\nthe labels against which we evaluate are often biased (unfairly against a protected attribute) in the\nvery same way as the unfair classi\ufb01er trained on them [11, 7, 3, 8, 20, 4, 14, 2, 13, 17]. It may be\nthe case that label-bias is so obvious to most authors that it does not even occur to them to mention\nit; howbeit, the conspicuous absence of label-bias from papers on fairness perniciously pervades\nreal-world discussions underlying the decisions about how to balance the trade-off between fairness\nand accuracy. Thus, we believe this \ufb01nding to be of practical importance and worthy of highlighting.\nWhile uncommon, some papers do indeed mention label-bias, including recent work that considers\nthe largely hypothetical case: if we have access to unbiased labels, then we can propose a better way\nof evaluating fairness with \u201cdisparate mistreatment\u201d [20]. However, their emphasis is on new fairness\nmetrics, not on its tradeoff with accuracy. Other work mentions the problem of label bias in passing,\nlamenting that it is dif\ufb01cult to account for in practice because we \u201conly have biased data\u201d and thus\nwe \u201ccannot evaluate our classi\ufb01ers against an unbiased ground truth\u201d and so achieving parity requires\n\n2\n\n\fthat \u201cone must be willing to reduce accuracy\u201d ([8]). They overcome the lack of unbiased labels via\ndata simulation, a strategy we also employ.\nCongruent with our \ufb01ndings, others have noted that the fairness-accuracy tension is not as bad as it\nseems. Recent work correctly remarks that while there is a tradeoff between fairness and goodness of\n\ufb01t on the training set, that \u201cit does not [necessarily] introduce a tension\u201d since a reduction in model\ncomplexity via fairness constraints might act as a regularizer and improve generalization [2]. This\nis a very interesting remark, but it could have gone even further and addressed generalization with\nrespect to the unbiased labels, which we study in this work.\nIn recent theoretical work, the authors\u2019 propose a \u201cconstruct space\u201d in which the observed data\nmight differ from some unobserved actual truth about the world [9]. While they investigate many\ndifferent notions of fairness, they do not address accuracy. The construct space provides a promising\ntheoretical framework for our work, but we save such analysis for another day. Other analytical\nwork studies the trade-off between fairness and accuracy as a function of the amount of statistical\ndependence between the target class and protected attribute, concluding that only \u201cin the other\nextreme\u201d of perfect independence that \u201cwe can have maximum accuracy and fairness simultaneously\u201d\n[17]. This \u201cextreme\u201d is none other than the \u201cwe\u2019re all equal\u201d assumption, which we believe to be\nperfectly reasonable in many situations. Further, note that this theoretical \u201cmaximum\u201d may not be\nachievable in practice due to imperfect classi\ufb01ers trained on incomplete, noisy data, or in the context\nof the phenomena mentioned herein, and hence there is still an opportunity to improve both.\nIt is worth thinking about the problems of selection and label bias with respect to an existing fairness\ndatasets such as COMPAS, for which the labels are sometimes treated as if they are the unbiased\nground truth [20]. Consider that the people in the COMPAS data had been selected from a speci\ufb01c\ncounty in Florida with its concomitant pattern of policing, during a speci\ufb01c period of time (2013-\n2014), meeting a speci\ufb01c set of criteria such as being scored during a speci\ufb01c stage within the judicial\nsystem. Each one of these \u201cselections\u201d opens the door for selection bias to introduce unintentional\ncorrelations. Indeed, recent work demonstrates that the data is skewed with respect to age, which\nacts as a confounding variable in existing analysis [18]. Moreover, while not exactly label-bias, the\nvariable indicating recidivism is only partially observed since it considers only a two-year window\nand assumes that no crime goes uncaught.\nFinally, we emphasize that our \ufb01ndings do not imply that the existing theories and conclusions\ndiscussed in the literature are incorrect. On the contrary, these works are in fact both sound and\nrelevant. The different conclusions then are explained by the consideration of different types of data\nbias (or lack thereof) as well as the underlying assumptions, and our assumptions may not always\napply [3]. If there differences between groups based on a protected attribute (e.g., due to selection\nbias), then enforcing fairness could indeed hurt accuracy. We do not address the degree to which\none assumption applies to a particular problem or dataset in this paper. Thus, just like in statistical\nsigni\ufb01cance testing, it remains up to the discretion of the discerning practitioner to determine if our\n(or their) set assumptions reasonably apply to the situation in question, and if the assumptions do not,\nthen our (or their) conclusions do not apply, and should be properly rejected as irrelevant to that data.\n\n3 Background\n\nFairness and bias types We consider two types of biases that lead to unfair machine learning\nmodels: label bias and selection bias. Label bias is when the observed binary class labels, say, on the\ntraining and testing set, are in\ufb02uenced by protected attributes. For example, the labels in the dataset\nmight be the result of yes/no hiring decisions for each job candidate. It is known that this hiring\nprocess is sometimes biased with respect to protected attributes such as race, age or gender. Since\ndecisions might be in\ufb02uenced by protected attributes that on the contrary should have no bearing on\nthe class label, this implies there is a hypothetical set of latent unobserved labels corresponding to\ndecisions that were not in\ufb02uenced by these attributes. We notate these unobserved unbiased labels as\nz. We notate the observed biased labels as y. Typically, we only have access to the latter for training\nand testing our models.\nSelection bias (skew) occurs when the method employed to select some subset of the overall pop-\nulation biases or skews the subset in unexpected ways. This can occur if selecting based on some\nattribute that inadvertently correlates with a protected class or the target labels. Training sets are\nparticularly vulnerable to such bias because, for the sake of manual labeling expedience, they are\n\n3\n\n\fmeager subsamples of the original unlabeled data points. Moreover, this problem is compounded\nsince most available labeled datasets are statically frozen in time and are thus also selectionally biased\nalong the axis of time. For example, in natural language processing (NLP), the particular topical\nsubjects or the entities mentioned in newswire articles change over time: the entities discussed in\npolitical discourse today are very different from a decade ago and new topics must emerge to keep\npace with the dernier cri [19]. And, as we continue to make progress in reducing discrimination, the\ndiscrepancy between the training data of the past and the available data of the present will increasingly\ndiffer w.r.t. to selection bias. Indeed, selection bias might manifest itself in a way such that on the\nrelatively small training set, the data examples that were selected for labeling happen to show bias\nagainst the protected class. It is with this manifestation of selection bias that we are most concerned,\nand that we study in the current work.\n\nIllustrative example: learning fair sectors Consider the problem of learning circular sectors of\nthe unit disk with the following attributes: the domain set X is the unit disk, the label set Y is {0, 1},\nthe data generation model D is an arbitrary density on X , the labeling function f is an arbitrary\npartition of X into two circular sectors, the hypothesis class H is the set of all partitions of X into two\ncircular sectors. Samples from D are points on the unit disk with location rei where 2 [0, 360)\nand r 2 [0, 1]. We represent a circular sector as a pair of angles (\u00b5, \u2713) and de\ufb01ned as the circular\nsector from angle (\u00b5 \u2713)%360 to angle (\u00b5 + \u2713)%360 that contains the point ei\u00b5. The labeling\nfunction f partitions the disk in two circular sectors f1(0) and f1(1) and we will refer to the\nformer as the negative circular sector and the latter as the positive circular sector. Note that for any\nlabeling function f, we have f 2H and so the realizable assumption holds.\nDue to label bias, the labeling function f might be biased (f0) as shown in Figure 1. Here, the\ntotal positive area according to f is given by the area in green and red, but because of label bias f0\nonly considers points in green as positive. Hence, as demonstrated in Figure 2, an empirical risk\nminimization (ERM) algorithm will learn a sector (dotted lines) that appears accurate with respect to\nf0, but is much less accurate with respect to f. If we had prior knowledge that the ratio of the positive\nsector and negative sector should be some constant k, perhaps we could exploit this and improve the\nERM solution. We might term such an alternative empirical fairness maximization (EFM) (or fair\nERM [5]), and in this paper, we present a semi-supervised EFM algorithm to exploit such fairness\nknowledge as a constraint on unlabeled data. This example is fully developed in appendix B.\n\ny\n\n(0, 1)\n\ny\n\n(0, 1)\n\n(1, 0)\n\n\u21e2\n\n(1, 0)\n\nx\n\n\u00b5\n\n(1, 0)\n\n(0,1)\nFigure 1\n\n(1, 0)\n\nx\n\n\u02c6\u21e2\n\n\u02c6\u00b5\n\n(0,1)\nFigure 2\n\nSemi-supervised classi\ufb01cation A binary classi\ufb01er3 gw : Rk !{ 0, 1} parameterized by a set of\nweights w 2 Rk is a function from a k dimensional real valued feature space, which is often in\npractice binary, to a binary class label. A probabilistic model pw(\u00b7|x) parameterized by (the very\nsame) w underlies the classi\ufb01er in the sense that we perform classi\ufb01cation by selecting the class label\n(0 or 1) that maximizes the conditional probability of the label y given the data point x\n\ngw(x) = argmax\ny2{0,1}\n\npw(y|x)\n\n(1)\n\n3For ease of explication, we consider the task of binary classi\ufb01cation, though our method can easily be\ngeneralized to multiclass classi\ufb01cation, multilabel classi\ufb01cation, or more complex structured prediction settings.\n\n4\n\n\fWe can then train the classi\ufb01er in the usual supervised manner by training the underlying model to\nassign high probability to each observed label yi in the training data Dtr = {hxi, yii| i = 1 . . . n}\ngiven the corresponding example xi, by minimizing the negative log likelihood:\n\n\u02c6w = argmin\n\nw2Rk Xhxi,yii2Dtr\n\n log pw(yi|xi)\n\n(2)\n\nWe can extend the above objective function to include unlabeled data Dun = {xi}n\ni=1 to make the\nclassi\ufb01er semi-supervised. In particular, we add a new term to the loss, C(Dun, w), with a weight \u2318 to\ncontrol the in\ufb02uence of the unlabeled data over the learned weights:\n\n\u02c6w = argmin\n\nw2Rk 0@ Xhxi,yii2Dtr\n\n log pw(yi|xi)1A + \u2318C(Dun, w)\n\nThe key question is how to de\ufb01ne the loss term C over the unlabeled data in such a way that improves\nover our classi\ufb01er.\n\n(3)\n\n(4)\n\n4 Approach\n\nApropos the foregoing discussion, we propose to employ fairness in the part of the loss function\nthat exploits the unlabeled data. There are of course many de\ufb01nitions of fairness proposed in the\nliterature that we could adapt for this purpose, but for now we focus on a particular type of group\nfairness constraint derived from the statistical parity of selection rates. Although this de\ufb01nition has\n(rightfully) been criticized, it has also (rightfully) been advocated in the literature and it underlies\nlegal de\ufb01nitions such as the 4/5ths rule in U.S. law [6, 8, 21]. For the purpose of this paper, we do\nnot wish to enter the fray on this particular matter.\nMore formally, let S = {xi}n\nclassi\ufb01er gw is \u00afgw(S) = 1\nand unprotected (DU\n\nnPxi2S gw(xi). If we partition our data (Dun) into the protected (DP\n\ni=1 be a set of n unlabeled examples, then the selection rate of the\nun)\n\nun, then we want the selection rate ratio\n\nun) partitions such that Dun = DP\nun [D U\n\u00afgw(DP\nun)\n\u00afgw(DU\nun)\n\nto be as close to one as possible. However, to make the problem more amenable to optimization via\nstochastic gradient descent, we relax this de\ufb01nition of fairness to make it differentiable with respect\nto w. In particular, analogous to \u00afgw(S), de\ufb01ne \u00afpw(S) = 1\nprobability of the set when assigning each example xi to the positive class yi = 1. Then, the group\nfairness loss over the unlabeled data \u2014 which when plugged into Equation 3 yields an instantiation\nof the proposed semisupervised training technique discussed herein \u2014 is\n\nnPxi2S pw(y = 1|xi) to be the average\n\nC(Dun, w) =\u00afpw(DP\n\n(5)\nParity is achieved at zero, which intuitively encodes that overall, the probability of assigning one\ngroup to the positive class should on average be the same as assigning the other group to the positive\nclass. This loss has the important property that it is differentiable with respect to w so we can optimize\nit with stochastic gradient descent, along with the supervised term of the objective, making it easy to\nimplement in existing toolkits such as Scikit-Learn, PyTorch or TensorFlow.\n\nun)2\nun) \u00afpw(DU\n\n5 Experiments\n\nIn this section we investigate the relationship between fairness and accuracy under conditions in\nwhich we can account for (and vary) the amount of label bias, selection bias, and the extent to which\nthe classi\ufb01ers enforce fairness. Typically, accuracy is measured against the ground truth labels on\nthe test set, which inconspicuously possesses the very same label bias as the training set. In this\ntypical evaluation setting, if we train a set of classi\ufb01ers that differ only in the extent to which their\ntraining objective functions enforce fairness, and then record their respective fairness and accuracy\nscores on a test set with such label bias, we see that increased fairness is achieved at the expense\n\n5\n\n\f(a) COMPAS (biased ground truth)\n\n(b) COMPAS (unbiased ground truth)\n\nFigure 3: Accuracy vs. fairness on simulated (=0.25) COMPAS (assumption hold).\n\nof accuracy (Figure 3a). However, because the labels are biased, we must immediately assume that\nthe corresponding accuracy measurements are also biased. Therefore, we are crucially interested\nin evaluating accuracy on the unbiased ground truth labels, which are devoid of any such label\nbias. Since we do not have access to the unbiased ground truth labels of real-world datasets, we\nmust instead rely upon data simulation. We discuss the details later, but for now, assume we could\nevaluate on such data. In Figure 3a, we evaluate the same set of classi\ufb01ers as before, but this time\nmeasure accuracy with respect to the unbiased ground truth labels. We see the exact opposite pattern:\nclassi\ufb01ers that are more fair are also more accurate. With the gist of our results and experimental\nstrategy in hand, we are now ready to describe the assumptions, data simulator, and systems to\nundertake a more comprehensive empirical investigation.\n\nAssumptions We make a set of assumptions that we encode directly into the probabilistic data\ngenerator, explained in more detail below. For example, we encode the \u201cwe\u2019re all equal assumption\u201d\nby making the unbiased labels statistically independent of the protected class [9]. If these assumptions\ndo not hold in a particular situation, then our conclusions may not apply. We describe the assumptions\nin more detail below and in the appendix.\n\nData Our experiments require datasets with points of the form D = {x, \u21e2, z, y} in which x is the\nvector of unprotected attributes, \u21e2 is the binary protected attribute, z is the (typically unobserved)\nlabel that has no label bias and y is the (typically observed) label that may have label bias. Since\nz is unobserved \u2014 and even if it were available, we would still want to vary the severity of label\nbias for experimental evaluation \u2014 we must rely upon data simulation [8]. We therefore assume\nthat the biased labels are generated from the unbiased labels via a probabilistic model g and assume\nthat y \u21e0 g(y|z, \u21e2, x, ) where is a parameter of the model that controls the probability of label\nbias occurring. Now we have two options for generating datasets of our desired form, we can either\n(a) simulate the dataset entirely from scratch from a probabilistic model of the joint distribution\nP (x, \u21e2, z, y) = g(y|z, \u21e2, x, )P (z, \u21e2, x)P (), or we can (b) begin with an existing dataset, declare\nby \ufb01at that the labels have no label bias (and are thus observed after all) and then augment the data\nwith a set of biased labels sampled from g(y|z, \u21e2, x, ).\nFor data of type (a) we generate the features and labels (both biased and unbiased) entirely from\nscratch with the Bayesian network in Figure 7 (Appendix A.2). For this data, we explicitly enforce\nthe following statistical assumptions: z, x ?? \u21e2, y 6?? \u21e2, z 6?? x, y 6?? z. A parameter controls the\namount of label bias; controls the amount of selection bias, which can break some assumptions.\nFor data of type (b) we begin with the COMPAS data, treat the two-year recidivism labels as\nthe unbiased ground-truth z and then apply our model of label bias to produce the biased labels\ny \u21e0 g(z|y, \u21e2, x, ) [15]. Since the \u201cwe\u2019re all equal\u201d assumption does not hold for COMPAS data we\nalso create a second type of test data in which we enforce demographic parity via subsampling so\nthat our assumption holds (see Appendix A.3).\n\nSystems, baselines and evaluations We study the behavior of the following classi\ufb01cation systems.\nA traditional supervised classi\ufb01er trained on biased label data, a supervised classi\ufb01er trained on\nunbiased label data (this in some sense is an ideal model, but not possible in practice because we do\nnot have access to the unbiased labels in practice), a random baseline in which labels are sampled\naccording to the biased label distribution in the training data, and three fair classi\ufb01ers. The \ufb01rst\nfair classi\ufb01cation method is an in-processing classi\ufb01er that employs our fairness constraint, but as\na regularizer on the training data instead of the unlabeled data. The resulting classi\ufb01er is similar\n\n6\n\n\f(a) F1 (unbiased truth)\nFigure 4: Classi\ufb01er accuracy (F1) and fairness as a function of the amount of label bias.\n\n(b) F1 (biased truth)\n\n(c) Fairness\n\n(a) F1 (unbiased truth)\n\n(b) F1 (biased truth)\n\n(c) Fairness\n\n(d) F1 (unbiased truth (v))\n\nFigure 5: Varying label bias on COMPAS (assumption holds, except in 5d).\n\nto the prejudice remover, but with a slightly different loss [12]. The second fair classi\ufb01er is a\nsupervised logistic regression trained using the \u201creweighing\u201d pre-processing method [11]. The \ufb01nal\nfair classi\ufb01er, which we introduce in this paper, is a semi-supervised classi\ufb01er that utilizes the fairness\nloss (Equation 5) on the unlabeled data.\nWe assess fairness with a group metric that computes the ratio of the selection rates of the protected\nand unprotected class, as we de\ufb01ned in Equation 4. A score of one is considered perfectly fair. To\nassess \u2018accuracy\u2019 we compute the weighted macro F1, which is the macro average weighted by the\nrelative portion of examples belonging to the positive and negative classes. We evaluate F1 with\nrespect to both the biased labels and the unbiased labels. We always report the mean and standard\nerror of these various metrics computed over ten experiments with ten randomly generated datasets\n(or in the case of COMPAS, ten random splits).\n\n5.1 Experiment 1: Label Bias\n\nIn this experiment we investigate the relationship between fairness and accuracy for each classi\ufb01cation\nmethod as we vary the amount of label bias. All classi\ufb01ers except the unbiased baseline are trained on\nbiased labels. If we evaluate the classi\ufb01ers on the biased labels as in Figure 4b (data simulated from\nscratch) or Figure 5b (COMPAS data) we see that the classi\ufb01ers that achieve high fairness (close to\none, as seen in Figure 4c&5c) sometimes degrade the (biased) F1 accuracy as commonly seen in the\nliterature. On the other hand, if we evaluate the classi\ufb01ers on the unbiased labels as in Figure 4a&5a,\nwe see that fairness and accuracy are in accord: the classi\ufb01ers that achieve high fairness achieve\nbetter accuracy than the fairness-agnostic supervised baseline. The gap between the fair and unfair\nclassi\ufb01ers increases as label bias increases. We also evaluate the classi\ufb01ers on COMPAS data that\nviolates the \u201cwe\u2019re all equal\u201d assumption. In this case, the fairness classi\ufb01ers are enforcing something\nuntrue about the data, and thus fairness initially degrades accuracy (Figure 5d). However, as the\namount of label bias increases, eventually there comes a point at which fairness once again improves\naccuracy (possibly because the amount of label bias exceeds the amount of other forms of bias).\n\n5.2 Experiment 2: Selection Bias\n\nWe repeat the experiment from the last section, but this time \ufb01xing label bias ( = 0.2) and subjecting\nthe training data to various amounts of selection bias by lowering the probability that a data example\nwith a positive label is assigned to the protected class. This introduces correlations in the training set\nbetween the protected class and the input features as well as correlations with both the unbiased and\n\n7\n\n\f(a) F1 (unbiased truth)\n\n(b) F1 (biased truth)\n\n(c) Fairness\n\nFigure 6: Classi\ufb01er accuracy (F1) and fairness as a function of the amount of selection bias.\n\nbiased labels. These correlations do not exist in the test set and unlabeled set which we assume do\nnot suffer from selection bias. We vary selection bias along the abscissa while keeping the label bias\nat a constant 20%, and report the same metrics as before. Results are in Figure 6. The main \ufb01ndings\nare that (a) the results are consistent with the theory that fairness and accuracy are in accord and (b)\nthat the semi-supervised method succesfully harnesses unlabeled data to correct for the selection and\nlabel bias in the training data (while the inprocessing fairness method succumbs to the difference in\ndata distribution between training and testing). Let us now look at these \ufb01ndings in more detail and\nin the context of the other baselines.\nInterestingly, the fairness-agnostic classi\ufb01ers and two of the fairness-aware classi\ufb01ers (in- and pre-\nprocessing) all succumb to selection bias, but in opposite ways (Figure 6c). The fairness-agnostic\nclassi\ufb01er learns the correlation between the protected attribute and the label and is unfair to the\nprotected class. In contrast, the two supervised fair classi\ufb01ers, for which fairness is enforced with\nstatistics of the training set both learn to overcompensate and are unfair to the unprotected class (its\nfairness curve is above 1). In both cases, as selection bias increases, so does unfairness and this results\nin a concomitant loss in accuracy (when evaluated not only against the unbiased labels (Figure 6a),\nbut also against the biased labels (Figure 6b)), indicating that fairness and accuracy are in accord.\nFinally, let us direct our attention to the performance of the proposed semi-supervised method by\nexamining the same \ufb01gures (Figure 6c). Now we see that regardless of the amount of selection bias,\nthe semi-supervised method successfully harnesses the unbiased unlabeled data to rectify it, as seen\nby the \ufb02at fairness curve achieving a nearly perfect 1 (Figure 6c). Moreover, this improvement in\nfairness over the supervised baseline (biased trained) is associated with a corresponding increase in\naccuracy relative to that same baseline (Figures 6a & 6b), regardless of whether it is evaluated with\nrespect to biased (20% label-bias) or unbiased labels (0% label-bias). Note that the \u201cwe\u2019re all equal\u201d\nassumption is violated as soon as we evaluate against the biased labels. Moreover, the label-bias\ninduces a correlation between the protected class and the target label, which is a common assumption\nfor analysis showing that fairness and accuracy are in tension [17]. Yet, the bene\ufb01cial relationship\nbetween accuracy and fairness is unsullied by the incorrect assumption in this particular case.\n\n6 Conclusion\n\nWe studied the relationship between fairness and accuracy while controlling for label and selection\nbias and found that under certain conditions the relationship is not a trade-off but rather one that\nis mutually bene\ufb01cial: fairness and accuracy improve together. We focused on demographic parity\nin this paper, but the ideas emphasized in this work, especially label bias, have potentially serious\nimplications for other notions of fairness that go beyond even their relationship with accuracy. In\nparticular, recent ways of assessing fairness such as disparate mistreatment, equal odds and equal\noppurtunity involve error rates as measured against labeled data. Label bias raises questions about the\nreliability of such measures and investigating such questions \u2014 about how label bias affects fairness\nand whether this causes fairness methods to undercompensate or overcompensate \u2014 is an important\ndirection of future work. Other future directions would be to develop more complex models of\nlabel and selection bias for particular domains so we can better understand the relationship between\nfairness and accuracy in these domains.\n\n8\n\n\f7 Acknowledgements\n\nWe thank the anonymous reviewers for their constructive feedback and helpful suggestions on how to\nstrengthen the paper.\n\nReferences\n[1] Maria-Florina Balcan and Avrim Blum. An augmented pac model for semi-supervised learn-\ning. In Olivier Chapelle, Bernhard Scholkopf, and Alexander Zien, editors, Semi-Supervised\nLearning, chapter 21. MIT Press, 2006.\n\n[2] Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie H.\nMorgenstern, Seth Neel, and Aaron Roth. A convex framework for fair regression. CoRR,\nabs/1706.02409, 2017.\n\n[3] Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism\n\nprediction instruments. CoRR, abs/1703.00056, 2016.\n\n[4] Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic\ndecision making and the cost of fairness. In Proceedings of the 23rd ACM SIGKDD International\nConference on Knowledge Discovery and Data Mining, KDD \u201917, pages 797\u2013806, New York,\nNY, USA, 2017. ACM.\n\n[5] Michele Donini, Luca Oneto, Shai Ben-David, John Shawe-Taylor, and Massimiliano Pontil.\nEmpirical risk minimization under fairness constraints. In Proceedings of the 32Nd International\nConference on Neural Information Processing Systems, NIPS\u201918, pages 2796\u20132806, USA, 2018.\nCurran Associates Inc.\n\n[6] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness\nthrough awareness. In Proceedings of the 3rd Innovations in Theoretical Computer Science\nConference, ITCS \u201912, pages 214\u2013226, New York, NY, USA, 2012. ACM.\n\n[7] Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkata-\nsubramanian. Certifying and removing disparate impact. In Proceedings of the 21th ACM\nSIGKDD International Conference on Knowledge Discovery and Data Mining, KDD \u201915, pages\n259\u2013268, New York, NY, USA, 2015. ACM.\n\n[8] Benjamin Fish, Jeremy Kun, and \u00c1d\u00e1m D\u00e1niel Lelkes. A con\ufb01dence-based approach for\n\nbalancing fairness and accuracy. In SDM, 2016.\n\n[9] Sorelle A. Friedler, Carlos Eduardo Scheidegger, and Suresh Venkatasubramanian. On the\n\n(im)possibility of fairness. CoRR, abs/1609.07236, 2016.\n\n[10] Kuzman Ganchev, Jo\u00e3o Gra\u00e7a, Jennifer Gillenwater, and Ben Taskar. Posterior regularization\n\nfor structured latent variable models. J. Mach. Learn. Res., 11:2001\u20132049, August 2010.\n\n[11] Faisal Kamiran and Toon Calders. Data preprocessing techniques for classi\ufb01cation without\n\ndiscrimination. Knowl. Inf. Syst., 33(1):1\u201333, October 2012.\n\n[12] Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. Fairness-aware learning through\nregularization approach. In Proceedings of the 2011 IEEE 11th International Conference on\nData Mining Workshops, ICDMW \u201911, pages 643\u2013650, Washington, DC, USA, 2011. IEEE\nComputer Society.\n\n[13] Jon Kleinberg. Inherent trade-offs in algorithmic fairness. SIGMETRICS Perform. Eval. Rev.,\n\n46(1):40\u201340, June 2018.\n\n[14] Jon M. Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair\n\ndetermination of risk scores. In ITCS, 2017.\n\n[15] Jeff Larson, Surya Mattu, Lauuren Kirchner, and Julia Angwin. ProPublica COMPAS data.\n\nhttps://github.com/propublica/compas-analysis, 2016.\n\n9\n\n\f[16] Gideon S. Mann and Andrew McCallum. Generalized expectation criteria for semi-supervised\n\nlearning with weakly labeled data. J. Mach. Learn. Res., 11:955\u2013984, March 2010.\n\n[17] Aditya Krishna Menon and Robert C Williamson. The cost of fairness in binary classi\ufb01cation. In\nSorelle A. Friedler and Christo Wilson, editors, Proceedings of the 1st Conference on Fairness,\nAccountability and Transparency, volume 81 of Proceedings of Machine Learning Research,\npages 107\u2013118, New York, NY, USA, 23\u201324 Feb 2018. PMLR.\n\n[18] Cynthia Rudin, Caroline Wang, and Beau Coker. The age of secrecy and unfairness in recidivism\n\nprediction. arXiv preprint arXiv:1811.00731, 2018.\n\n[19] Xuerui Wang and Andrew McCallum. Topics over time: A non-markov continuous-time model\nof topical trends. In Proceedings of the 12th ACM SIGKDD International Conference on\nKnowledge Discovery and Data Mining, KDD \u201906, pages 424\u2013433, New York, NY, USA, 2006.\nACM.\n\n[20] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi.\nFairness beyond disparate treatment & disparate impact: Learning classi\ufb01cation without\ndisparate mistreatment. In Proceedings of the 26th International Conference on World Wide Web,\nWWW \u201917, pages 1171\u20131180, Republic and Canton of Geneva, Switzerland, 2017. International\nWorld Wide Web Conferences Steering Committee.\n\n[21] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi.\n\nFairness constraints: Mechanisms for fair classi\ufb01cation. In AISTATS, 2017.\n\n10\n\n\f", "award": [], "sourceid": 4734, "authors": [{"given_name": "Michael", "family_name": "Wick", "institution": "Oracle Labs"}, {"given_name": "swetasudha", "family_name": "panda", "institution": "Oracle Labs"}, {"given_name": "Jean-Baptiste", "family_name": "Tristan", "institution": "Oracle Labs"}]}