Synthetic-Real Experience Oscillation: A Dynamic Hybrid Training Paradigm

TL;DR: What if we rhythmically alternate between pure synthetic and pure real experiences during training? We hypothesize this oscillation could prevent overfitting to either domain and boost sim-to-real transfer, testing this by DreamGym agents in WebArena with varying oscillation frequencies.

Research Question: How does periodic switching between synthetic-only and real-only experience phases affect agent robustness and adaptation during curriculum learning?

Hypothesis: Controlled oscillation between synthetic and real experiences creates "cognitive dissonance" that forces agents to develop domain-agnostic representations, improving generalization compared to continuous hybrid training (as used in DreamGym).

Experiment Plan: - Setup: Extend DreamGym’s framework to implement three training regimes: (a) continuous hybrid (baseline), (b) synthetic-real oscillation (switch every N episodes), and (c) random phase switching.

• Data: Use WebArena and CARLA environments; track task success rates, domain shift metrics (e.g., feature covariance differences), and adaptation speed.
• Analysis: Compare performance on held-out real-world tasks; measure forgetting via replay buffer probes.
• Expected Outcome: Oscillation regimes (especially with N tuned via meta-learning) should show 15–20% higher real-world task transfer than DreamGym’s static hybrid approach.

References: ['Chen, Z., et al. (2025). Scaling Agent Learning via Experience Synthesis. arXiv.org.', 'Jin, B., & Guo, W. (2024). Synthetic Social Media Influence Experimentation Via an Agentic Reinforcement Learning Large Language Model Bot. Journal of Artificial Societies and Social Simulation.', 'Lu, C., et al. (2023). Synthetic Experience Replay. Neural Information Processing Systems.']