Robust Trust Region Methods for Corrupted Multi-Agent Systems

Kuba et al. (2021) develop HATRPO/HAPPO, ensuring monotonic policy improvement in MARL. But this assumes all agents follow the algorithm. Tsuchiya et al. (2024) show this is unrealistic—agents may deviate due to faults or malice. This idea asks: Can trust region methods be made robust to corruption? We’d design a robust HATRPO where each agent’s policy update accounts for worst-case deviations from others (e.g., using corruption bounds). Technically, we’d integrate Tsuchiya et al.’s adaptive learning rates into Kuba et al.’s sequential update scheme, ensuring joint policies improve despite noise. This would be critical for safety-critical systems (e.g., autonomous vehicle coordination) where some agents might be compromised. The novelty lies in merging optimality (trust regions) with robustness (corruption dynamics).

References:

1. Corrupted Learning Dynamics in Games. Taira Tsuchiya, Shinji Ito, Haipeng Luo (2024). Annual Conference Computational Learning Theory.
2. Trust Region Policy Optimisation in Multi-Agent Reinforcement Learning. J. Kuba, Ruiqing Chen, Munning Wen, Ying Wen, Fanglei Sun, Jun Wang, Yaodong Yang (2021). International Conference on Learning Representations.