Qualitative Co-Design of Alignment: Bringing Users into the Loop for Interpretability and Trustworthiness

Most personalization research (e.g., "User Embedding Model for Personalized Language Prompting" by Doddapaneni et al., 2024; "ExploreLLM" by Ma et al., 2023) focuses on implicit or data-driven profiling. However, Ben Wang et al. (2024) highlight the benefits of integrating user perceptions and qualitative input to improve engagement and satisfaction. This research proposes a new paradigm: participatory, qualitative co-design sessions where users articulate their alignment goals, concerns, and edge cases directly (using structured interviews, scenario mapping, and interactive prototyping). The LLM alignment process then incorporates these qualitative insights as explicit constraints or objectives, and returns explanations or visualizations of how user input shapes model behavior. This approach foregrounds interpretability and user trust, especially for underrepresented or vulnerable user groups, and could help bridge the gap between technical optimization and real-world acceptability of personalized LLMs.

References:

1. Task Supportive and Personalized Human-Large Language Model Interaction: A User Study. Ben Wang, Jiqun Liu, Jamshed Karimnazarov, Nicolas Thompson (2024). Conference on Human Information Interaction and Retrieval.
2. User Embedding Model for Personalized Language Prompting. Sumanth Doddapaneni, Krishna Sayana, Ambarish Jash, Sukhdeep S. Sodhi, D. Kuzmin (2024). PERSONALIZE.
3. Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses. Xiao Ma, Swaroop Mishra, Ariel Liu, S. Su, Jilin Chen, Chinmay Kulkarni, Heng-Tze Cheng, Quoc V. Le, E. Chi (2023). CHI Extended Abstracts.