Curriculum Design by Difficulty Forecasting: Anticipatory Adaptive Environment Sequencing for RL-LMs

Research Question: Does anticipatory curriculum sequencing, based on forecasting model learning plateaus, accelerate and stabilize LM training in adaptive procedural RL environments?

Hypothesis: By forecasting when a model is approaching mastery in a current difficulty regime and preemptively introducing more challenging tasks, we can prevent learning stagnation and maintain a high learning signal, resulting in faster and more stable improvements.

Experiment Plan: - Implement a meta-controller atop RLVE-Gym that fits simple models (e.g., logistic regression) to recent episode reward histories to predict near-future performance plateaus.

• When a plateau is forecasted, automatically escalate environment difficulty or introduce new environment types.
• Compare learning progress, convergence speed, and benchmark performance to RLVE’s standard adaptive difficulty mechanism.
• Analyze whether anticipatory adaptation prevents “dead zones” of little to no learning.

References: ['Zeng, Z. et al. (2025). RLVE: Scaling Up Reinforcement Learning for Language Models with Adaptive Verifiable Environments.', 'Nesterova, M., Skrynnik, A., & Panov, A. (2024). Adaptive Curriculum Learning: Optimizing Reinforcement Learning through Dynamic Task Sequencing. Optical Memory and Neural Networks.']