Originally published on Tumblr.

Kiki and the Mathematical Impossibility of Fairness

TL;DR: No classifier can satisfy all fairness constraints simultaneously, as proven by Choquet’s theorem and impossibility theorems.

Fairness in AI is a mathematical mirage.

The quest for fairness in AI systems often encounters a paradoxical barrier: the mathematical impossibility of satisfying multiple fairness constraints simultaneously. This is not just a theoretical quibble but a profound limitation grounded in the very structure of statistical decision-making. Choquet’s theorem and various impossibility theorems, such as those concerning equalized odds, demographic parity, and calibration, illustrate that no classifier can achieve all these fairness metrics at once. The implications are stark: efforts to enforce fairness in one dimension can inadvertently exacerbate unfairness in another.

Consider the statistical tools we use to measure fairness: confusion matrices, precision-recall curves, and ROC-AUC scores. These metrics reveal the disparate impact of classifiers across different demographic groups. For instance, a classifier optimized for equalized odds might ensure that true positive rates are equal across groups, but this often comes at the cost of demographic parity, where the overall selection rates differ. Similarly, calibration—where predicted probabilities reflect actual outcomes—can conflict with both equalized odds and demographic parity.

• Disparate Impact: Confusion matrices show how different groups experience varying rates of false positives and false negatives.
• Precision-Recall Curves: These highlight trade-offs between precision and recall, often revealing biases in how different groups are treated.
• ROC-AUC Scores: While useful for assessing overall classifier performance, these scores can mask underlying disparities between groups.

Bayes-optimal classifiers, which are designed to minimize error rates, inherently perpetuate base rate differences between groups. This is because they are fundamentally aligned with existing statistical distributions, which often reflect societal biases. Algorithmic fairness interventions, therefore, tend to shift discrimination from one metric to another rather than eliminating it. This was starkly illustrated in a recent AI funding bubble, where overpromised capabilities led to failed projects that couldn’t reconcile these fairness constraints.

In the end, the pursuit of fairness in AI requires more than just technical solutions; it demands a societal reckoning with the biases embedded in our data. As we continue to develop AI systems, we must ask ourselves: are we willing to accept the trade-offs inherent in algorithmic fairness, or should we strive for deeper systemic changes that address the root causes of inequality?

For those interested in the technical nuances, I recommend diving deeper into the statistics of disparate impact and the limitations of current fairness metrics. The journey is complex, but understanding these challenges is crucial for developing truly equitable AI systems.

Tags: mathematical-impossibility, choquet-theorem, fairness-constraints, disparate-impact, algorithmic-fairness, bayes-optimal-classifiers, demographic-parity, equalized-odds, calibration, ai-bias