When Fairness Flips: Detecting, Explaining, and Preempting Context-Driven Bias Reversals in Deployed Health AI

Yang et al. (2024) show that collaborative models can diverge sharply across HIC/LMIC sites, and Wang et al. (2023) report a striking pattern in readmission prediction: White and higher-income groups have higher FNR, while Black and low-income groups have higher FPR and 0–1 loss—with substantial heterogeneity across hospitals. This project proposes a fairness “control tower” that (a) continuously tracks group-wise metrics across sites, (b) uses change-point detection to flag unexpected reversals (e.g., subgroup A’s FPR rising above subgroup B’s after deployment), and (c) applies a structural causal model to decompose those flips into components attributable to care access/usage patterns (as cautioned by Wang et al., 2023), model specification, and target mismatch. We explicitly incorporate Tal’s (2023) target specification bias by stress-testing models under counterfactual labels that better reflect decision-makers’ intended target (e.g., “would-be readmission if post-discharge support were provided”). This differs from existing post-hoc fairness audits by treating “flip events” as primary signals to investigate mechanisms, rather than averaging fairness over time and sites. The payoff is operational: health systems can preemptively re-calibrate or re-define targets when fairness flips are driven by utilization or target mismatch, rather than bluntly debiasing models in ways that may harm utility or equity. If successful, this yields safer cross-hospital deployment protocols with fewer unintended disparities and better generalizability across HIC/LMIC contexts (Yang et al., 2024).

References:

1. Mitigating machine learning bias between high income and low–middle income countries for enhanced model fairness and generalizability. Jenny Yang, Lei A. Clifton, N. Dung, N. Phong, L. Yen, Doan Bui Xuan Thy, A. Soltan, Louise Thwaites, David A. Clifton (2024). medRxiv.
2. Evaluating Algorithmic Bias in 30-Day Hospital Readmission Models: Retrospective Analysis. H. E. Wang, Jonathan P. Weiner, S. Saria, Hadi Kharrazi (2023). Journal of Medical Internet Research.
3. Target specification bias, counterfactual prediction, and algorithmic fairness in healthcare. Eran Tal (2023). AAAI/ACM Conference on AI, Ethics, and Society.