Prompt Lattices for Consistency and Cross-Lingual Conductivity

Consistency failures—different answers for minor instruction rewrites—remain an open robustness issue (Yukun et al., 2024). CLiKA (Gao et al., 2024) reveals that multilingual pretraining and instruction tuning only shallowly improve cross-lingual knowledge alignment, especially conductivity. DAPT (Cho et al., 2023) and recent semantic alignment work for VLMs (Kuchibhotla et al., 2025) suggest that arranging embeddings improves alignment. Building on these, we propose prompt lattices: a shared, low-dimensional prompt subspace with tied parameters across paraphrase groups and translations, coupled via group-equivariant constraints that encourage invariance to surface form. The system stitches together retrieval-augmented, modular prompts (Feng et al., 2024) to inject domain exemplars at lattice “nodes,” and aligns across modalities using dual-aligned prompt signals where relevant (Hu et al., 2023; M^2PT by Wang et al., 2024). Novelty: rather than learning one prompt per task, we learn a structured set of prompts with explicit coupling that penalizes divergence across semantically equivalent instructions/languages. Impact: stronger instruction-following consistency under rephrasings, deeper cross-lingual conductivity (beyond performance on a single language), and more reliable multilingual assistants.

References:

1. Retrieval-Augmented Modular Prompt Tuning for Low-Resource Data-to-Text Generation. Ruitao Feng, Xudong Hong, Mayank Jobanputra, Mattes Warning, Vera Demberg (2024). International Conference on Language Resources and Evaluation.
2. Semantic Alignment for Prompt-Tuning in Vision Language Models. Hari Chandana Kuchibhotla, Sai Srinivas Kancheti, Abbavaram Gowtham Reddy, Vineeth N. Balasubramanian (2025). Trans. Mach. Learn. Res..
3. M^2PT: Multimodal Prompt Tuning for Zero-shot Instruction Learning. Taowen Wang, Yiyang Liu, J. Liang, Junhan Zhao, Yiming Cui, Yuning Mao, Shaoliang Nie, Jiahao Liu, Fuli Feng, Zenglin Xu, Cheng Han, Lifu Huang, Qifan Wang, Dongfang Liu (2024). Conference on Empirical Methods in Natural Language Processing.
4. Distribution-Aware Prompt Tuning for Vision-Language Models. Eulrang Cho, Jooyeon Kim, Hyunwoo J. Kim (2023). IEEE International Conference on Computer Vision.
5. Context-Aware Prompt Tuning for Vision-Language Model with Dual-Alignment. Hongyu Hu, Tiancheng Lin, Jie Wang, Zhenbang Sun, Yi Xu (2023). arXiv.org.
6. Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly. Changjiang Gao, Hongda Hu, Peng Hu, Jiajun Chen, Jixing Li, Shujian Huang (2024). North American Chapter of the Association for Computational Linguistics.
7. Improving the Robustness of Large Language Models via Consistency Alignment. Zhao Yukun, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiang Wang, Meng Chong, Zhicong Cheng, Zhaochun Ren, Yin Dawei (2024). International Conference on Language Resources and Evaluation.