Semantic Fidelity Control: Adaptive Abstraction Levels in LLM-Based Experience Synthesis for RL

Research Question: Can dynamically modulating the semantic abstraction level of LLM-generated experiences—ranging from high-level summaries to fine-grained state/action details—improve RL agent performance and generalization during both synthetic training and sim-to-real transfer?

Hypothesis: Adaptive abstraction, optimized per training phase or task complexity, will yield more efficient learning and better generalization compared to using a fixed abstraction level in synthetic experience generation.

Experiment Plan: Extend DreamGym’s experience model to generate rollouts at multiple semantic abstraction levels (inspired by the HTML simplification in AutoWebGLM and semantic compression in Ruichen Zhang et al.). Develop a controller that adjusts abstraction in response to learning signals (e.g., plateauing performance triggers more detail). Train agents on a suite of environments, comparing fixed vs. adaptive abstraction strategies for learning speed, final performance, and transfer success. Analyze which abstraction levels are most effective at different RL stages.

References: ['Lai, H., Liu, X., Iong, I. L., Yao, S., Chen, Y., Shen, P., Yu, H., Zhang, H., Zhang, X., Dong, Y., & Tang, J. (2024). AutoWebGLM: A Large Language Model-based Web Navigating Agent. Knowledge Discovery and Data Mining.', 'Zhang, R., Zhao, C., Du, H., Niyato, D., Wang, J., Sawadsitang, S., Shen, X., & Kim, D. I. (2025). Embodied AI-Enhanced Vehicular Networks: An Integrated Large Language Models and Reinforcement Learning Method. arXiv.org.']