Reversal-Consistent Knowledge Injection: DLM-Guided Paraphrastic Monte Carlo for Data-Scarce AR Fine-Tuning

TL;DR: Have a diffusion model rewrite each fact in many different orders so the AR model learns to answer questions no matter how you ask them. First experiment: use a DLM to generate forward/backward paraphrases of knowledge triples and fine-tune an AR model; hypothesis is closing the AR–DLM gap on backward QA with 5–10× less augmentation.

Research Question: Can DLM-generated paraphrastic Monte Carlo augmentation eliminate the AR “reversal curse” during knowledge injection and drastically improve AR data efficiency?

Hypothesis: Because DLMs learn any-order mappings and generalize to reversed styles without paraphrases, they can serve as targeted paraphrase generators that cover orderings AR models struggle with. Fine-tuning AR models on DLM-generated paraphrases should approach DLM performance on both forward and backward QA, reducing augmentation demands.

Experiment Plan: - Teacher/Generator: Train or adapt a DLM (e.g., DiffuLLaMA) on the target domain with limited unique data.

• Augmentation: For each fact or QA pair, sample diverse paraphrases spanning orderings and surface forms by controlling diffusion guidance and clamping; optionally control quality/diversity trade-off using classifier-free guidance analogs.
• Fine-Tuning: Fine-tune AR LMs on a small paraphrase budget (e.g., 2–5 per fact) and compare to standard AR with heavy paraphrase augmentation.
• Evaluation: Use forward/backward QA benchmarks from Pan et al.; measure accuracy, sample efficiency, and generalization to unseen styles.
• Expected: AR models fine-tuned with DLM-generated paraphrases achieve high backward QA accuracy with much less data, effectively closing the gap with dLLMs on knowledge generalization.

References: 1. Pan, X., Hahami, E., Fan, J., Xie, Z., & Sompolinsky, H. (2025). Closing the Data-Efficiency Gap Between Autoregressive and Masked Diffusion LLMs.
2. Ni, J., Liu, Q., Dou, L., Du, C., Wang, Z., Yan, H., Pang, T., & Shieh, M. (2025). Diffusion Language Models are Super Data Learners.
3. Gong, S., Agarwal, S., Zhang, Y., Ye, J., Zheng, L., Li, M., An, C., Zhao, P., Bi, W., Han, J., Peng, H., & Kong, L. (2024). Scaling Diffusion Language Models via Adaptation from Autoregressive Models. International Conference on Learning Representations.
4. Buzzard, Z. (2025). Understanding the Quality-Diversity Trade-off in Diffusion Language Models. arXiv.org.