Anomaly-Triggered Curriculum Learning for Multi-Agent Coordination

Building on Li et al.’s [2024] work on embedding RL in behavior trees and their discovery of unexpected interruptions, this idea takes the anomaly one step further: rather than masking or mitigating it, why not use it as a signal for adaptive learning? Here, when an anomaly (e.g., breakdown in coordination, performance dip) is detected during training or deployment, the system automatically generates new, targeted training scenarios that exaggerate or probe the anomaly. This creates a dynamic curriculum where agents are repeatedly exposed to their own failure modes, gradually mastering more complex and subtle coordination problems. Unlike standard curriculum learning, which is often hand-designed, this approach is self-adaptive and specifically tuned to the agents’ emergent weaknesses. This could lead to much faster convergence and more robust team behaviors, especially in large, unpredictable environments like robot soccer (see MARLadona [Li et al., 2024]). The impact: agents that continuously self-improve by confronting and overcoming their own emergent coordination failures.

References:

1. Embedding multi-agent reinforcement learning into behavior trees with unexpected interruptions. Xianglong Li, Yuan Li, Jieyuan Zhang, Xinhai Xu, Donghong Liu (2024). Complex & Intelligent Systems.
2. MARLadona - Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning. Zichong Li, Filip Bjelonic, Victor Klemm, Marco Hutter (2024). IEEE International Conference on Robotics and Automation.