Learning to Learn Interestingness: Meta-RL for Self-Improving Mathematical Discovery Agents

Research Question: Can meta-reinforcement learning be leveraged to enable automated theory formation agents to autonomously refine their own models of mathematical interestingness over time?

Hypothesis: Meta-RL will allow FERMAT or similar agents to surpass static or hand-crafted interestingness functions, dynamically adapting to new domains and evolving a more nuanced, context-aware sense of mathematical curiosity.

Experiment Plan: - Implement a meta-RL framework within FERMAT that treats the interestingness function itself as a policy to be optimized.

• The outer loop meta-learner updates the interestingness scoring mechanism based on the downstream success (e.g., human ratings, theorems reused in further discoveries, or publication-worthiness) of generated theories.
• Benchmark against baseline static and evolutionary interestingness measures.
• Measure adaptability to new mathematical domains and the capacity to escape local optima in theory formation.

References:

• Surina, A., Mansouri, A., Quaedvlieg, L., Seddas, A., Viazovska, M., Abbe, E., & Gulcehre, C. (2025). Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning. arXiv.org.
• Pease, A., Colton, S., & Charnley, J. (2013). Automated Theory Formation: The Next Generation.