The Paradox of Explanations: When Human Rationales Hinder OOD Generalization

Most literature assumes explanations and rationales are always beneficial (see ER-Test; Chen et al., 2023), but this may not hold if rationales encode environment-specific or spurious correlations (as cautioned by Liu et al., 2024, and Lin et al., 2023). This project would construct datasets and benchmarks where human rationales are not invariant across environments, perhaps due to annotator bias or context-specific reasoning. By training models with and without rationale-based regularization, the research would empirically test if and when explanations degrade OOD performance. This challenges a core assumption and could lead to guidelines for when (and how) to use rationales, possibly developing algorithms that selectively ignore explanations in certain regimes. The outcome would be a more nuanced understanding of the role of explanations in generalization, avoiding one-size-fits-all prescriptions.

References:

1. Understanding and Improving Feature Learning for Out-of-Distribution Generalization. Yongqiang Chen, Wei Huang, Kaiwen Zhou, Yatao Bian, Bo Han, James Cheng (2023). Neural Information Processing Systems.
2. Spurious Feature Diversification Improves Out-of-distribution Generalization. Yong Lin, Lu Tan, Yifan Hao, Honam Wong, Hanze Dong, Weizhong Zhang, Yujiu Yang, T. Zhang (2023). International Conference on Learning Representations.
3. ER-Test: Evaluating Explanation Regularization Methods for Language Models. Brihi Joshi, Aaron Chan, Ziyi Liu, Shaoliang Nie, Maziar Sanjabi, Hamed Firooz, Xiang Ren (2022). Conference on Empirical Methods in Natural Language Processing.