Consensus-Driven Reward Structuring for Resolving Cooperation-Convergence Conflicts

Feng et al. [2024] introduce hierarchical consensus to guide agent cooperation without direct communication, while Sidahmed & Chavdarova [2024] highlight convergence issues rooted in rotational optimization dynamics. This idea synthesizes these strands by using consensus signals—not only to guide actions but also to dynamically reshape each agent’s reward function in real time. If agents' local policies conflict or show signs of non-convergent dynamics, the system can automatically increase the weight of consensus-based reward components, nudging the population toward more stable cooperation. Conversely, in well-coordinated states, the reward can shift back toward individual or local objectives. This adaptive reward structuring could help resolve the chronic tension between individual reward maximization and group convergence, especially in settings with unreliable communication (cf. Jiang et al., 2024). The novelty is in leveraging consensus not just for action selection, but as a lever for reward shaping—potentially stabilizing learning in notoriously difficult MARL domains.

References:

1. Hierarchical Consensus-Based Multi-Agent Reinforcement Learning for Multi-Robot Cooperation Tasks. Pu Feng, Junkang Liang, Size Wang, Xin Yu, Rongye Shi, Wenjun Wu (2024). IEEE/RJS International Conference on Intelligent RObots and Systems.
2. Addressing Rotational Learning Dynamics in Multi-Agent Reinforcement Learning. Baraah A. M. Sidahmed, Tatjana Chavdarova (2024).
3. A Model-Based Reinforcement Learning Algorithm for Multi-Agent Cooperation Nash Equilibrium With Unstable Communication. Yuannan Jiang, Shengming Jiang, Xiaofeng Wang (2024). IEEE Transactions on Circuits and Systems - II - Express Briefs.