Designing RLVR-Native Parameter-Efficient Fine-Tuning: Beyond SFT-Era Heuristics

Research Question: Can we create parameter-efficient fine-tuning (PEFT) techniques explicitly tailored to the geometric and spectral characteristics of RLVR updates, and do these outperform SFT-era PEFT methods?

Hypothesis: RLVR-native PEFT modules that restrict updates to spectrum-preserving, off-principal directions will yield better reasoning gains per parameter and less spectral drift than SFT-derived LoRA or sparse adapters.

Experiment Plan: - Design new adapter layers or update rules that target low-curvature, off-principal subspaces (inspired by the Three-Gate Theory).

• Compare these RLVR-native adapters to SFT-era LoRA and sparse adapters on reasoning benchmarks.
• Measure spectral drift, principal subspace rotation, and reasoning performance.
• Expected result: RLVR-native PEFT modules outperform SFT-era modules in efficiency and maintain model stability.

References: ['Zhu, H., Zhang, Z., Huang, H., Su, D., Liu, Z., Zhao, J., Fedorov, I., Pirsiavash, H., Sha, Z., Lee, J., Pan, D. Z., Wang, Z., Tian, Y., & Tai, K. S. (2025). The Path Not Taken: RLVR Provably Learns Off the Principals.', 'Wu, F., Xuan, W., Qi, H., Lu, X., Tu, A., Li, L. E., & Choi, Y. (2025). DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search. arXiv.org.']