Cross-Architecture Generalization: RL Agents that Learn to Optimize for Future, Unseen GPU Hardware

TL;DR: What if we could train RL agents to generate CUDA kernels that run fast not only on today’s GPUs but also on future, unseen architectures? The experiment would involve “leave-one-architecture-out” training and testing, measuring how well the agent anticipates new hardware quirks.

Research Question: Can RL-based CUDA kernel optimizers trained on multiple GPU architectures generalize to deliver high performance on entirely new, previously unseen hardware designs?

Hypothesis: Exposure to a diverse portfolio of hardware feedback during RL training will enable agents to internalize fundamental optimization principles, leading to robust performance on novel GPU architectures.

Experiment Plan: - Setup: Train RL agents on CUDA kernel optimization tasks across a variety of NVIDIA and AMD GPUs.

• Test: Hold out latest or rare architectures for testing only.
• Data: Use KernelBench and real-world kernels; collect profiler data across hardware.
• Metrics: Measure speedup, correctness, and code portability.
• Expected Outcome: Cross-architecture–trained RL agents will significantly outperform single-architecture agents and compilers on unseen hardware.

References:

• 1. Li, X., Sun, X., Wang, A., Li, J., & Shum, C. (2025). CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning. arXiv.org.
• 2. Dong, K. S., Modi, S., Nikiforov, D., Damani, S., Lin, E., Hari, S., & Kozyrakis, C. (2026). KernelBlaster: Continual Cross-Task CUDA Optimization via Memory-Augmented In-Context Reinforcement Learning.