Infinity-Chat++: Real-Time, Longitudinal Tracking of Model Homogeneity in the Wild

Research Question: How do model updates, fine-tuning cycles, and shifting user populations dynamically affect intra- and inter-model homogeneity and diversity in open-ended outputs?

Hypothesis: Model updates and user-base shifts cause measurable, sometimes unpredictable, changes in the degree of output homogeneity, with occasional “diversity shocks” following major releases or dataset changes.

Experiment Plan: Launch a web platform where open-ended queries are continually collected from real users and run through multiple LMs (including evolving versions). Track output diversity metrics and annotate with user demographics, model version, and update history. Analyze temporal trends: do diversity dips/stagnations correspond to model updates, training data changes, or user population shifts? Test interventions (e.g., injecting novel prompts or adversarial data) to see if they can disrupt developing homogeneity.

References:

• Jiang, L., Chai, Y., Li, M., Liu, M., Fok, R., Dziri, N., Tsvetkov, Y., Sap, M., Albalak, A., & Choi, Y. (2025). Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond). arXiv.org.