Hybrid Symbolic-Neural Test-Time RL for Enhanced Program Synthesis

TL;DR: Why not combine the best of both worlds? Let’s let LLMs use symbolic solvers or search as tools during test-time RL, creating a hybrid system for program synthesis in open problems.

Research Question: Can a hybrid symbolic-neural architecture, where LLMs collaborate with symbolic solvers at test time, lead to more sample-efficient and creative solutions to open optimization problems?

Hypothesis: Hybrid systems will find better or more diverse solutions with fewer samples by leveraging the strengths of both neural generation and symbolic reasoning during test-time adaptation.

Experiment Plan: Integrate a symbolic solver (e.g., constraint programming toolkit) into ThetaEvolve’s RL loop. The LLM proposes high-level strategies or code fragments; the symbolic solver performs local refinement or verification. Reward the LLM based on the improvement achieved by the hybrid approach. Compare against pure LLM-based RL and pure symbolic solvers in terms of solution quality, diversity, and speed.

References:

• 1. Wang, Y., et al. (2025). ThetaEvolve: Test-time Learning on Open Problems.
• 2. Aguina-Kang, R., Gumin, M., Han, D. H., Morris, S., Yoo, S. J., Ganeshan, A., Jones, R. K., Wei, Q. A., Fu, K., & Ritchie, D. (2024). Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases. arXiv.org.