Counterfactual Reasoning Chains: Improving LLM Skepticism via Stepwise Justification

TL;DR: Make models explain every step of their reasoning, so they can “catch themselves” before agreeing with bad medical info. The experiment would require LLMs to generate explicit reasoning chains and cross-check each step against known safety rules or medical knowledge.

Research Question: Does forcing LLMs to produce explicit, stepwise justifications for their medical QA outputs increase their resistance to accepting counterfactual or harmful evidence?

Hypothesis: Requiring LLMs to decompose their reasoning into traceable steps, and automatically checking these steps for safety violations, will uncover more counterfactual inconsistencies and reduce unsafe outputs compared to end-to-end generation.

Experiment Plan: - Adapt the MedCounterFact dataset to include a “reasoning chain required” format.

• Fine-tune or prompt LLMs to output multi-step explanations.
• Automatically analyze each reasoning step for contradiction with established safety knowledge (e.g., using knowledge graphs or curated medical rules).
• Compare error and harm rates with standard QA generation.
• Conduct ablation studies to identify which reasoning steps are most vulnerable to counterfactual contamination.

References:

• Mo, K. et al. (2026). Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence.
• Wu, J., Deng, W., Li, X., Liu, S., Mi, T., Peng, Y., et al. (2025). MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs. arXiv.org.