To Be Fair: Interrogating Fairness through Blindness in Fair Representation Learning

Abstract

As machine learning based classification tools become increasingly prevalent and consequential, developing techniques to ensure these models work fairly and ethically is vital. Traditional fair learning approaches which often entail satisfying group level statistical criteria have been criticized for leading to unfair outcomes on the individual level. Fair representation learning (FRL) is a subfield of fair machine learning which attempts to achieve individual fairness by producing representations which obfuscate a given sensitive characteristic, s, while retaining as much information as possible about the target task. However, FRL research often evaluates for an overly narrow view of fairness. This paper expands the scope of evaluation for two FRL models, incorporating a broader set of group fairness metrics and interrogating FRL’s own notion of fairness more deeply. These two models are the Domain Adversarial Neural Network (DANN) with origins in domain adaptation, and the FarconVAE. The DANN does a poor job of enforcing obfuscation of s in its latent representations, which may reflect a broader limitation in the applicability of domain adaptation based approaches to FRL. The FarconVAE effectively obfuscates s without making significant sacrifices to accuracy on the target task. However, classifiers trained on these latent representations show notable changes in several group fairness metrics, some which improve fairness outcomes and some which reduce fairness. These results reinforce the importance of incorporating a broad view of fairness, grounded in a continued engagement with moral and ethical considerations, into the development of fair learning techniques.