Rubric-Driven Skill Internalization: Scaffolding Exploration for LLM Agents

TL;DR: What if we used explicit rubrics or checklists as scaffolding during skill internalization, helping the agent explore “how” and “why” behind each skill, rather than just “what” to do?

Research Question: Does providing rubric-based scaffolding during RL for skill internalization (à la RuscaRL) lead to deeper, more flexible reasoning and transferable skills in LLM agents?

Hypothesis: Rubric-scaffolded RL will result in agents that better understand underlying skill rationales, leading to improved transfer, interpretability, and robustness compared to SKILL0’s visual/contextual skill grouping alone.

Experiment Plan: For each skill, design explicit rubrics/checklists (inspired by Zhou et al., 2025) to guide agent exploration and provide structured feedback during RL training. Compare rubric-guided vs. standard SKILL0 curriculum on reasoning and manipulation tasks. Analyze transfer to out-of-domain or composite tasks, and evaluate interpretability (e.g., LLM-generated explanations). Expectation: Rubric guidance yields more compositional and transparent skill acquisition.

References:

• Zhou, Y., Li, S., Liu, S., Fang, W., Zhao, J., Yang, J., Lv, J., Zhang, K., Zhou, Y., Lu, H., Chen, W., Xie, Y., & Song, M. (2025). Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoning. arXiv.org.
• Lu, Z., Yao, Z., Wu, J., Han, C., Gu, Q., Cai, X., Lu, W., Xiao, J., Zhuang, Y., & Shen, Y. (2026). SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalization.