Disentangling Procedural and Declarative Knowledge in RL-Tuned Language Models

Research Question: How does reinforcement learning fine-tuning differentially affect procedural versus declarative knowledge representations in LLMs, and can we causally disentangle their neural signatures?

Hypothesis: RL enhances procedural (route-finding) abilities in hierarchical knowledge traversal without substantially altering declarative (fact-storage) representations, and these can be independently manipulated within the model.

Experiment Plan: Design two sets of evaluation tasks: (a) Pure fact recall (e.g., “What is code 57.95?”), (b) Procedural recall (e.g., “List the path from root to code 57.95 in the hierarchy”). Apply targeted activation ablation or “knockout” interventions (e.g., via masking or feature suppression) at layers identified in Zhang et al. (2025) as showing divergence. Measure performance drop on each task type when ablating “procedural” vs. “factual” subspaces. Use representational similarity analysis and causal mediation to confirm separation. If successful, this would provide the field with a road map of “where” and “how” RL alters LLM cognition.

References: ['Zhang, R., Kaniselvan, M., & Mireshghallah, N. (2025). Reinforcement Learning Improves Traversal of Hierarchical Knowledge in LLMs.', 'Wang, W., Tan, A., Teow, L.-N., & Tan, Y.-S. (2014). Declarative-procedural memory interaction in learning agents. Adaptive Agents and Multi-Agent Systems.']