Outcome-driven Reverse Curriculum RL for Organizational Protocol Discovery

TL;DR: Suppose AsyncThink could “teach itself” how to organize agents by working backwards from successful outcomes using only final result feedback. Test curriculum learning where the protocol learns efficient sub-task assignment from good end states, not just explicit process supervision.

Research Question: Can outcome-only supervision with curriculum learning efficiently train agentic organizers to discover novel, efficient sub-task assignment and aggregation strategies?

Hypothesis: Reverse curriculum RL (Xi et al., 2024) will allow agentic organizations to bootstrap efficient organizational strategies from successful outcomes, even in the absence of dense step-by-step supervision—outperforming conventionally RL-trained organizers where detailed process feedback is sparse.

Experiment Plan: Build an RL training loop for AsyncThink organizers using outcome-only (final answer correct/incorrect) feedback. Use reverse curriculum learning: start by training on sub-problems that are close to the solution, then progressively train on earlier, less constrained organization decisions. Evaluate on multi-step reasoning and tool use benchmarks (cf. Chai et al., 2025; Xi et al., 2024). Compare learning efficiency and final accuracy to process-supervised RL and static heuristics.

References: ['Zhiheng Xi, Wenxiang Chen, Boyang Hong, Senjie Jin, Rui Zheng, Wei He, Yiwen Ding, Shichun Liu, Xin Guo, Junzhe Wang, Honglin Guo, Wei Shen, Xiaoran Fan, Yuhao Zhou, Shihan Dou, Xiao Wang, Xinbo Zhang, Peng Sun, Tao Gui, Qi Zhang, Xuanjing Huang. (2024). Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning. International Conference on Machine Learning.', 'Jiajun Chai, Guojun Yin, Zekun Xu, Chuhuai Yue, Yi Jia, Siyu Xia, Xiaohan Wang, Jiwen Jiang, Xiaoguang Li, Chengqi Dong, Hang He, Wei Lin. (2025). RLFactory: A Plug-and-Play Reinforcement Learning Post-Training Framework for LLM Multi-Turn Tool-Use. arXiv.org.']