Disentangling Race and Task: A Sparse Autoencoder Approach for Generalizable LLM Debiasing

Nguyen and Tan's 2025 paper identifies 'race subspaces' in large language models (LLMs) and intervenes on them to reduce bias, but these subspaces do not generalize well across varying prompts, indicating entanglement with specific tasks. This research proposes applying Sparse Autoencoders (SAEs) to decompose these race subspaces into sparse, interpretable features, distinguishing between 'pure' race-related concepts and 'task-entangled' concepts. By targeting only the SAE features causally linked to biased outcomes—drawing inspiration from Chen et al.'s 'FAST' knowledge editing—this approach aims for more precise bias mitigation that generalizes better across prompts and tasks. This method challenges the assumption of a stable, single race representation and instead treats it as a dynamic composition of underlying features, potentially enabling debiasing interventions that preserve model performance and align with the 'no harm' principle advocated by Zhu et al.

References:

1. On the Effectiveness and Generalization of Race Representations for Debiasing High-Stakes Decisions. Dang Nguyen, Chenhao Tan (2025). arXiv.org.
2. Large Language Model Bias Mitigation from the Perspective of Knowledge Editing. Ruizhe Chen, Yichen Li, Zikai Xiao, Zuo-Qiang Liu (2024). arXiv.org.
3. Do Not Harm Protected Groups in Debiasing Language Representation Models. Chloe Qinyu Zhu, Rickard Stureborg, Brandon Fain (2023). arXiv.org.