Principia-XL: Multilingual and Multimodal Extensions for Structured Mathematical Reasoning

TL;DR: What if we could make models just as good at deriving mathematical objects in Korean, Russian, or with visual diagrams as they are in English? The experiment would extend the Principia suite to multiple languages and modalities, assessing models’ cross-lingual and multimodal reasoning abilities.

Research Question: How well do current and newly trained LLMs generalize structured mathematical reasoning to multilingual and multimodal contexts, and what training or architectural changes are needed to close any observed gaps?

Hypothesis: Test-time aggregation and on-policy reward modeling approaches will face new challenges in non-English and multimodal settings, revealing unique error patterns and suggesting the need for language- and modality-specific adaptation strategies.

Experiment Plan: - Translate and adapt Principia tasks to at least five languages and include multimodal (text+diagram) variants.

• Evaluate leading LLMs (e.g., Qwen, o3, DeepSeekMath, MindOmni) with the same aggregation and on-policy judge training methods.
• Analyze language- and modality-specific failure modes.
• Experiment with curriculum learning and synthetic data augmentation to improve cross-lingual performance.

References:

• Son, G., Hong, J., Ko, H., & Thorne, J. (2025). Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning. Annual Meeting of the Association for Computational Linguistics.
• Oh, S., Park, J., & Baek, H. (2026). Performance Evaluation of OpenAI’s o4-mini on CSAT Mathematics: Multimodal Reasoning and the Reasoning–Verification–Reanalysis Loop. IEEE Access.