Negative Transfer-Aware Meta-Learning for Continual Reinforcement Learning

Ahn et al.'s (2024) Reset & Distill method reveals that negative transfer is a critical but overlooked problem in continual RL. This research extends their work by incorporating meta-learning to proactively identify when transfer will be harmful. The system would meta-learn a transfer evaluator that: (1) estimates task similarity in representation space, (2) predicts potential interference between current and new tasks, and (3) decides between knowledge distillation (as in R&D) versus network reset. This challenges the norm in meta-RL (like Marini et al.'s UAV trajectory optimization) that transfer is always beneficial. By learning transfer boundaries, we could achieve the efficiency gains of CoMPS while avoiding the performance drops from negative transfer—something current meta-RL methods don't address.

References:

1. Reset & Distill: A Recipe for Overcoming Negative Transfer in Continual Reinforcement Learning. Hongjoon Ahn, Jinu Hyeon, Youngmin Oh, Bosun Hwang, Taesup Moon (2024). arXiv.org.
2. Continual Meta-Reinforcement Learning for UAV-Aided Vehicular Wireless Networks. Riccardo Marini, Sangwoo Park, O. Simeone, C. Buratti (2022). ICC 2023 - IEEE International Conference on Communications.