Dynamic Diversity-Driven Hybrid Search for Automated AI Research

TL;DR: What if we let the system switch between evolutionary search and RL based on real-time diversity signals, so it avoids getting “stuck” on boring ideas? Concretely, an experiment could compare adaptive hybrid schedules—triggered by entropy or novelty metrics—against static or single-method baselines in automated LLM research ideation.

Research Question: Can an adaptive hybrid framework that dynamically alternates between evolutionary search and RL (using online diversity metrics) sustain high-quality and diverse idea generation in execution-grounded LLM research?

Hypothesis: Dynamically switching between evolutionary and RL phases—when diversity or entropy drops—will counteract early saturation and mode collapse, outperforming static or single-method search in both upper-bound and average research outcomes.

Experiment Plan: Build on the automated executor setup from Si et al. (2026). Monitor diversity metrics (e.g., policy entropy, solution novelty, f-divergence from initial ideator) during search. When diversity substantially drops, switch from RL to evolutionary search (or vice versa); optionally, introduce hybrid crossover/mutation during transitions. Compare against (a) pure RL, (b) pure evolutionary, (c) static hybrid schedules on benchmarks like LLM post-training and pre-training idea search. Key measurements: diversity (entropy, unique solution count), best/average reward, and scaling trends over epochs.

References:

• Si, C., Yang, Z., Choi, Y., Candès, E. J., Yang, D., & Hashimoto, T. (2026). Towards Execution-Grounded Automated AI Research.
• Li, L., Hao, J., Liu, J. K., Zhou, Z., Tan, X., Chu, W., Wang, Z., Pan, S., Qu, C., & Qi, Y. (2025). The Choice of Divergence: A Neglected Key to Mitigating Diversity Collapse in Reinforcement Learning with Verifiable Reward. arXiv.org.
• Chen, J., Liu, F., Liu, N., Luo, Y., Qin, E., Zheng, H., Dong, T., Zhu, H., Meng, Y., & Wang, X. (2025). Step-wise Adaptive Integration of Supervised Fine-tuning and Reinforcement Learning for Task-Specific LLMs. arXiv.org.