Multi-Agent Synthetic Worlds: Scaling Reasoning-Based Experience Synthesis for Cooperative and Competitive RL

Research Question: How well can reasoning-based LLM environment models simulate complex multi-agent interactions—including cooperation, competition, and negotiation—and what are the limits of scaling DreamGym to multi-agent RL domains?

Hypothesis: Extending DreamGym to multi-agent settings, with LLMs simulating diverse agent behaviors, will enable scalable training of robust multi-agent policies, but may expose new challenges in stability and emergent behavior fidelity.

Experiment Plan: Augment DreamGym to simulate multiple agents with diverse policies (inspired by cMALC-D’s LLM-guided curriculum for MARL). Benchmark on cooperative (e.g., traffic signal control) and competitive (e.g., adversarial games) environments. Measure sample efficiency, policy robustness, and emergent behaviors compared to standard MARL baselines. Analyze synthetic social interactions for realism and generalization to real multi-agent scenarios.

References: ['Satheesh, A., Powell, K., & Wei, H. (2025). cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending. Proceedings of the 34th ACM International Conference on Information and Knowledge Management.', 'Jin, B., & Guo, W. (2024). Synthetic Social Media Influence Experimentation Via an Agentic Reinforcement Learning Large Language Model Bot. Journal of Artificial Societies and Social Simulation.']