Diversity-Preserving Self-Evolution through Controlled Novelty Injection

TL;DR: What if we explicitly inject novelty into the Challenger-Solver loop to prevent the model from becoming too repetitive or overconfident? Inspired by EVOL-RL and the entropy collapse problem, this approach randomly seeds a fraction of Challenger generations with prompts or reasoning chains drawn from out-of-domain or adversarial examples. The experiment would test whether this “novelty injection” slows or prevents diversity collapse over long iterations.

Research Question: Does deliberately injecting diverse or adversarial prompts into the Challenger’s generation pipeline maintain diversity and reasoning complexity during self-evolution, without destabilizing learning?

Hypothesis: Controlled novelty injection will counteract diversity collapse and enhance reasoning robustness, yielding higher pass@n and out-of-domain generalization compared to standard R-Few.

Experiment Plan: - Modify R-Few’s Challenger to sample a portion of prompts from a pool of out-of-domain or model-disagreement cases (as in Zhou et al., 2025).

• Track entropy of responses, diversity of reasoning chains, and pass@n metrics.
• Evaluate on both in-domain and cross-domain benchmarks (e.g., math + general QA).
• Expected outcome: Models with novelty injection maintain higher diversity and generalize better.

References:

• 1. Zhou, Y., Liang, Z., Liu, H., Yu, W., Panaganti, K., Song, L., Yu, D., Zhang, X., Mi, H., & Yu, D. (2025). Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation. arXiv.org.
• 2. Herel, D., & Mikolov, T. (2024). Collapse of Self-trained Language Models. Tiny Papers @ ICLR.
• 3. [R-Few paper cited as source]