Simulation-to-Real Transfer with Safety-Driven Domain Randomization

While Liu et al. (2024) note the limitations of simulation-trained safe policies, current sim2real transfer often neglects the fact that real-world safety constraints are uncertain or change over time. My proposal: employ domain randomization not just for dynamics, but for constraints themselves—randomizing constraint surfaces, violation costs, and even the nature of penalties during simulation training. This forces the agent to develop robust, adaptive safety strategies that generalize to unknown or shifting constraints in the real world. It’s a synthesis of sim2real philosophy with constraint learning, going beyond the fixed-manifold or hard-masking strategies in existing work. The anticipated impact? RL agents that are much safer and more adaptable when transitioning from simulator to the messy, unpredictable real world.

References:

1. Safe Reinforcement Learning on the Constraint Manifold: Theory and Applications. Puze Liu, Haitham Bou-Ammar, Jan Peters, Davide Tateo (2024). IEEE Transactions on robotics.