Induction Meets Deduction: Iterative Schema Generation for Interpretable RL in Non-Markovian Environments

Building on Zhang et al. (2025)’s HRL-ID (neural-symbolic RL) and Foster & Jones (2017)’s analogical schema induction, this research would develop an RL architecture that alternates between inductive learning from trajectories (e.g., extracting temporal motifs or reward patterns in non-Markovian tasks, as per Tang et al. 2024) and deductive reasoning (e.g., enforcing logical consistency or temporal logic constraints). The system would iteratively refine symbolic schemas—temporal rules, automata, or analogies—that explain and guide agent behavior. Unlike prior work, this approach explicitly targets interpretable and transferable policy representations in environments where standard state-action abstractions are insufficient. Theoretical analysis would address convergence properties, while empirical studies would demonstrate improved generalization and transparency. This could bridge the gap between black-box deep RL and the need for trustworthy, explainable AI in safety-critical domains.

References:

1. Beyond Simple Sum of Delayed Rewards: Non-Markovian Reward Modeling for Reinforcement Learning. Yuting Tang, Xin-Qiang Cai, Jing-Cheng Pang, Qiyu Wu, Yao-Xiang Ding, Masashi Sugiyama (2024). arXiv.org.
2. Deduction with Induction: Combining Knowledge Discovery and Reasoning for Interpretable Deep Reinforcement Learning. Haodi Zhang, Xiangyu Zeng, Junyang Chen, Yuanfeng Song, Rui Mao, Fangzhen Lin (2025). International Joint Conference on Artificial Intelligence.
3. Reinforcement Learning with Analogical Similarity to Guide Schema Induction and Attention. J. Foster, Matt Jones (2017). arXiv.org.