Resampling Meets Knowledge Graphs: Integrating Structured Causal Walks with Distributional Reasoning in LLMs

TL;DR: What if you combined the idea of resampling many CoTs with structured random-walks over knowledge graphs, so you could see how often the model "walks" through the same causal pathways or finds new ones? We’ll compare the effects of KG-guided prompting vs. vanilla resampling on reasoning coverage and answer faithfulness.

Research Question: Does integrating KG-based random-walk reasoning with CoT resampling produce richer and more faithful distributions of LLM reasoning, and can this hybrid approach surface rare but correct causal explanations?

Hypothesis: KG-guided CoT resampling uncovers more diverse and accurate reasoning trajectories, especially for knowledge-intensive or multi-hop causal tasks, than either method alone.

Experiment Plan: Use benchmarks from commonsense question answering and causal reasoning. For each prompt, generate CoTs by (a) vanilla resampling and (b) KG-random-walk-guided resampling (per Kim et al., 2024). Analyze diversity, correctness, and overlap of reasoning chains; identify unique, rare, or especially insightful explanations. Measure if hybrid sampling reduces “unfaithful” reasoning or uncovers overlooked causal paths. Expected outcome: KG-guided sampling expands the set of plausible reasoning paths, improves answer robustness, and surfaces explanations missed by standard resampling.

References:

• 1. Yejin Kim, Kang, E., Kim, J., & Huang, H. H. (2024). Causal Reasoning in Large Language Models: A Knowledge Graph Approach. arXiv.org. 2. Debjit Paul, West, R., Bosselut, A., & Faltings, B. (2024). Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning. Conference on Empirical Methods in Natural Language Processing. 3. Guangsheng Bao, Zhang, H., Yang, L., Wang, C., & Zhang, Y. (2024). LLMs with Chain-of-Thought Are Non-Causal Reasoners. arXiv.org.