Neuro-Inspired Adaptive Gain Control for RL Exploration-Exploitation via Online Physiological Feedback

Küçükoğlu et al. (2022) show the power of predictive processing in RL, inspired by neuroscience. Yet, adaptive gain theory from cognitive neuroscience (Jepma & Nieuwenhuis, 2011) suggests biological agents tune their exploratory behavior using physiological cues like pupil diameter. This idea proposes the first RL algorithms that incorporate online physiological feedback—whether from real humans (in human-in-the-loop RL) or simulated neuro-inspired proxies—to modulate exploration rates or policy stochasticity. The research would formalize the connection between physiological arousal, uncertainty, and adaptive learning rates within RL, potentially yielding new theoretical guarantees about sample efficiency and robustness. Experiments would test these algorithms in complex, partially observable tasks (as in Pham et al., 2024), measuring both behavioral and physiological data. This could open an entirely new field at the intersection of RL, neuroscience, and human-computer interaction, grounding RL exploration strategies in biological principles.

References:

1. Adaptive Compensation for Robotic Joint Failures Using Partially Observable Reinforcement Learning. Tan-Hanh Pham, Godwyll Aikins, Tri Truong, Kim-Doang Nguyen (2024). Algorithms.
2. Efficient Deep Reinforcement Learning with Predictive Processing Proximal Policy Optimization. Burcu Küçükoğlu, Walraaf Borkent, Bodo Rueckauer, Nasir Ahmad, Umut Güçlü, M. Gerven (2022). Neurons, Behavior, Data analysis, and Theory.
3. Pupil Diameter Predicts Changes in the Exploration–Exploitation Trade-off: Evidence for the Adaptive Gain Theory. M. Jepma, S. Nieuwenhuis (2011). Journal of Cognitive Neuroscience.