Challenging the "Stable Preference" Assumption: Modeling and Exploiting Preference Volatility in LLM Alignment

Many alignment and personalization strategies—such as those in "Teaching Language Models to Evolve with Users" (Zhao et al., 2025) or "Unified Preference Optimization" (Badrinath et al., 2024)—implicitly assume that user preferences change gradually and can be captured with incremental updates. This research challenges that core assumption, proposing a volatility-aware framework where LLMs not only track user preferences but also model their degree and pattern of change (preference volatility). By introducing time-series modeling techniques (e.g., regime-switching models, volatility clustering), the LLM can anticipate abrupt shifts, quickly recalibrating alignment strategies. This is particularly relevant for users undergoing major life transitions, or in high-change environments (e.g., youth learning, crisis response). The innovation lies in treating volatility itself as a first-class modeling target, enabling LLMs to remain relevant and accurate even when user behavior is erratic or rapidly changing.

References:

1. Teaching Language Models to Evolve with Users: Dynamic Profile Modeling for Personalized Alignment. Weixiang Zhao, Xingyu Sui, Yulin Hu, Jiahe Guo, Haixiao Liu, Biye Li, Yanyan Zhao, Bing Qin, Ting Liu (2025). arXiv.org.
2. Unified Preference Optimization: Language Model Alignment Beyond the Preference Frontier. Anirudhan Badrinath, Prabhat Agarwal, Jiajing Xu (2024). Trans. Mach. Learn. Res..