Error-Driven Curriculum Learning for Multi-Agent LLMs

TL;DR: Let's teach multi-agent LLMs by focusing on their hardest mistakes first, so they get better at what they're worst at. Concretely, we could design a curriculum that adaptively prioritizes failed or challenging reasoning trajectories in the experience library, hypothesizing that this "mistake-first" progression will foster faster and more robust self-improvement.

Research Question: Can an error-driven curriculum, where multi-agent LLMs are explicitly trained on their most challenging or failed trajectories first, accelerate learning and yield more robust reasoning and negotiation skills compared to random or success-focused replay?

Hypothesis: Prioritizing unsuccessful or high-error trajectories in the training curriculum will lead to greater and more sample-efficient improvements in agent performance, especially in complex or adversarial multi-agent tasks.

Experiment Plan: Develop an adaptive curriculum scheduler that ranks and selects failed or low-reward trajectories for focused replay and refinement. Compare SiriuS-style experience library training using: 1. Success-first (as in SiriuS) 2. Error-driven (mistake-first) 3. Random sampling (baseline). Evaluate on reasoning-heavy benchmarks (e.g., GSM8K, negotiation) and track convergence speed, final performance, and generalization to unseen task types. Analyze learning dynamics, especially in early training.

References:

• Zhao, W., Yuksekgonul, M., Wu, S., & Zou, J. (2025). SiriuS: Self-improving Multi-agent Systems via Bootstrapped Reasoning. arXiv.org.
• Huang, Q., Luo, Y., Liu, Z., Xia, J., Chang, M., & Li, J. (2025). A Hierarchical Reinforcement Learning Framework for Multi-Agent Cooperative Maneuver Interception in Dynamic Environments. Journal of Marine Science and Engineering.