Longitudinal Trust Calibration: How Users Adapt to Shifting LM Beliefs Over Time

Research Question: How do users perceive and adapt to belief shifts in language models during extended, real-world interactions, and what factors influence their trust calibration?

Hypothesis: Users will initially overlook subtle LM belief shifts, but exposure to inconsistencies or explicit explanations will prompt recalibration of trust—potentially leading to increased skepticism, selective reliance, or disengagement.

Experiment Plan: Recruit users for multi-session interactions with an LM, with sessions designed to subtly or overtly induce belief shifts. Use self-report, behavioral metrics, and physiological proxies (e.g., fNIRS if available, as in Eloy et al., 2022) to track trust and reliance over time. Introduce transparency interventions (e.g., belief-shift notifications, explanations) and measure their effect on trust calibration. Analyze correlations between LM belief volatility, user demographics, task domain, and trust trajectories.

References: 1. Geng, J., Chen, H., Liu, R., Horta Ribeiro, M., Willer, R., Neubig, G., & Griffiths, T. L. (2025). Accumulating Context Changes the Beliefs of Language Models. 2. Eloy, L., Doherty, E., Spencer, C. A., Bobko, P., & Hirshfield, L. M. (2022). Using fNIRS to Identify Transparency- and Reliability-Sensitive Markers of Trust Across Multiple Timescales in Collaborative Human-Human-Agent Triads. Frontiers in Neuroergonomics.