Label-Robustness Testing and Mitigation via Controlled Language-ID Noise and Embedding Smoothing

Introduce systematic perturbations of language-ID labels during ASR inference to evaluate model sensitivity to label errors across languages and dialects. Develop a lightweight inference-time module that replaces hard language tags with weighted mixtures of language embeddings based on predicted language probabilities, inspired by Whisper-style embedding interpolation, to improve robustness especially for zero-shot dialects and unseen language varieties.

References:

1. Enhancing Multilingual ASR for Unseen Languages via Language Embedding Modeling. Shao-Syuan Huang, Kuan-Po Huang, Andy T. Liu, Hung-yi Lee (2024). arXiv.org.
2. Improving Multilingual ASR Robustness to Errors in Language Input. Brady Houston, Omid Sadjadi, Zejiang Hou, Srikanth Vishnubhotla, Kyu J. Han (2024). Interspeech.