Multi-Modal Uncertainty Distillation: Bridging Reasoning Robustness Across Text, Audio, and Visual Modalities

TL;DR: Can we use what we've learned about uncertainty in text to help LLMs reason better with images or audio? Design a multimodal distillation protocol that aligns uncertainty signals across modalities, then examine improvements in cross-modal OOD reasoning.

Research Question: How does the suppression or encouragement of epistemic uncertainty during self-distillation affect reasoning robustness in cross-modal (text, audio, vision) LLMs, and can multi-granularity alignment of uncertainty signals improve OOD generalization?

Hypothesis: Aligning and preserving epistemic uncertainty signals across different modalities during self-distillation will yield more robust multimodal reasoning, especially under domain shifts.

Experiment Plan: - Setup: Extend the CORD approach (Hu et al., 2026) to include explicit alignment and preservation of epistemic markers between text, audio, and visual reasoning traces during distillation.

• Data: Use tasks where the same concepts are expressed across modalities (e.g., VQA, audio QA, text-only reasoning).
• Measurements: Assess cross-modal OOD accuracy, uncertainty calibration, and transferability of robust reasoning behaviors.
• Expected Outcomes: Models trained with multimodal uncertainty alignment will show improved OOD performance, compared to unimodal or uncertainty-suppressing baselines.

References:

• Kim, J., Luo, X., Kim, M., Lee, S., Kim, D., Jeon, J., Li, D., & Yang, Y. (2026). Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?
• Hu, J., Zhu, D., Luo, X., Zhang, D., He, S., Lei, Y., Zheng, H., Feng, S., He, J., Sun, Y., Wu, H., & Wang, H. (2026). CORD: Bridging the Audio-Text Reasoning Gap via Weighted On-policy Cross-modal Distillation. arXiv.org.