Quantifying Curiosity as a Dynamic Intrinsic Reward in LLM Research Agents

Curiosity can be operationalized as a measurable score based on prediction error, uncertainty, novelty, or variance, and integrated into LLM agents to improve exploration and problem-solving.

Research Question: Can curiosity be reliably quantified as a dynamic intrinsic reward in LLM-based research agents, and does integrating this score improve exploration, robustness, and creativity?

Hypothesis: Curiosity—operationalized as a dynamic, intrinsic reward based on deviations from agent expectations or model uncertainty—can be reliably quantified as a curiosity score, and its integration will enhance exploration, robustness, and creative problem-solving in LLM agents.

Experiment Plan: Develop metrics for curiosity based on prediction error, uncertainty, novelty, and variance. Integrate these as intrinsic rewards during training alongside extrinsic rewards. Evaluate agent performance in sparse-reward and open-ended research tasks, measuring exploration efficiency, robustness, and creativity. Analyze curiosity scores during evaluation to identify knowledge gaps and unexpected behaviors.

References: Pan, Y., Liu, Z., & Wang, H. (2025). Wonder Wins Ways: Curiosity-Driven Exploration through Multi-Agent Contextual Calibration. arXiv.org.
Dai, R., Song, L., Liu, H., Liang, Z., Yu, D., Mi, H., Tu, Z., Liu, R., Zheng, T., Zhu, H., & Yu, D. (2025). CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models. arXiv.org.
Colas, C., Karch, T., Lair, N., Dussoux, J.-M., Moulin-Frier, C., Dominey, P. F., & Oudeyer, P.-Y. (2020). Language as a Cognitive Tool to Imagine Goals in Curiosity-Driven Exploration. Neural Information Processing Systems.
Niu, X., Ito, A., & Nose, T. (2024). A Replaceable Curiosity-Driven Candidate Agent Exploration Approach for Task-Oriented Dialog Policy Learning. IEEE Access.
Sergeant-Perthuis, G., Ruet, N., Ognibene, D., Tisserand, Y., Williford, K., & Rudrauf, D. (2025). Action of the Euclidean versus projective group on an agent’s internal space in curiosity driven exploration. Biological Cybernetics.