Selective Dispersion Steering via Dispersion Adapters for Improved Model Capabilities

Instead of global dispersion increase, train dispersion adapters that push away only factors with positive causal effects on perplexity while pulling close or constraining factors that harm consolidation or fairness. The training objective blends selective push-away, contrastive pull-close constraints, and regularizers for knowledge consolidation, fairness, and controlled forgetting, enabling safer and more effective model fine-tuning.

References:

1. On the Predictive Power of Representation Dispersion in Language Models. Yanhong Li, Ming Li, Karen Livescu, Jiawei Zhou (2025). arXiv.org.
2. Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Models. Lingzhi Wang, Xingshan Zeng, Jinsong Guo, Kam-Fai Wong, Georg Gottlob (2024). AAAI Conference on Artificial Intelligence.
3. Bias and Fairness in Large Language Models: Evaluation and Mitigation Techniques. Murali Krishna Pasupuleti (2025). International Journal of Academic and Industrial Research Innovations(IJAIRI).