Active Skill Diversification with Reward Mutation During Internalization

TL;DR: What if, instead of just internalizing the “canonical” skill, we encouraged the agent to discover variants by mutating reward functions during RL, leading to a richer internal skill set?

Research Question: Can actively mutating reward functions during skill internalization foster the emergence of diverse and adaptable skill variants in LLM agents, outperforming single-policy internalization as in SKILL0?

Hypothesis: Reward mutation will help agents internalize not just the intended skill, but a spectrum of related behaviors, boosting generalization and robustness for novel or unexpected tasks.

Experiment Plan: During RL-based skill internalization, periodically mutate reward functions (e.g., as in van Buuren et al., 2025) by adding noise or emphasizing different aspects (speed, accuracy, novelty). Analyze resulting policy diversity and ability to handle transfer or perturbation tasks. Compare to the standard SKILL0, which may converge to a single, optimal policy per skill. Expected: Mutated reward strategies yield a more flexible skill embedding.

References:

• van Buuren, J., Giglio, R., Roveda, L., & Peternel, L. (2025). Robotic Skill Diversification via Active Mutation of Reward Functions in Reinforcement Learning During a Liquid Pouring Task. arXiv.org.
• Lu, Z., Yao, Z., Wu, J., Han, C., Gu, Q., Cai, X., Lu, W., Xiao, J., Zhuang, Y., & Shen, Y. (2026). SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization.