Conflict-Aware Curriculum for Multi-Task RL Agents in Enterprise Search

TL;DR: Teach search agents by first identifying which tasks or search domains "fight" each other during training, then design a smart curriculum to avoid confusion—just like a teacher who spaces out hard-to-learn topics. The experiment would involve monitoring conflict signals (e.g., negative transfer, gradient interference) across KARLBench task types and scheduling training order adaptively to minimize such conflicts.

Research Question: How can we detect and mitigate task conflicts during multi-task reinforcement learning for knowledge agents, and does a conflict-aware training curriculum lead to better generalization and stability?

Hypothesis: Explicitly identifying and managing conflicts between diverse search tasks (e.g., entity retrieval vs. procedural reasoning) during training will reduce negative transfer and improve both in-domain and out-of-domain performance compared to naive multi-task training.

Experiment Plan: - Train multi-task agents on KARLBench, logging task-wise gradients and loss interference metrics (as in Li & Jia, 2025).

• Use these signals to cluster tasks based on conflict and design an adaptive curriculum that sequences or batches tasks to minimize interference.
• Compare conflict-aware curriculum agents to standard multi-task and random curriculum baselines on KARLBench, measuring convergence speed, generalization, and robustness to new (OOD) tasks.
• Analyze if certain task pairs consistently conflict and whether conflict mitigation mechanisms (e.g., task-specific adapters, gradient surgery) help.
• Success is measured by improved average and worst-case OOD task performance.

References:

• 1. Chang, J. D., Drozdov, A., Toshniwal, S., Oertell, O., Trott, A., Portes, J., Gupta, A., Koppol, P., Baheti, A., Kulinski, S., Zhou, I., Dea, I., Opsahl-Ong, K., Favreau-Lessard, S., Owen, S., Gonzalez Ortiz, J. J., Singhvi, A., Andrade, X., Wang, C., Sreenivasan, K. K., Havens, S., Liu, J., Deniro, P., Sun, W., Bendersky, M., Frankle, J. (2026). KARL: Knowledge Agents via Reinforcement Learning.
• 2. Li, K., & Jia, Q.-S. (2025). Multi-Agent Reinforcement Learning With Decentralized Distribution Correction. IEEE Transactions on Automation Science and Engineering.