Beyond Accuracy: Detecting and Explaining Model Failures in Multimodal Clinical and Educational NLP Systems

Anna Shalini et al. (2025) highlight that transformer models for multimodal sentiment analysis in healthcare and education lack interpretability, especially when their predictions go awry. Most XAI research emphasizes explaining correct predictions, but little attention is paid to systematically identifying and explaining failures or outliers (see also Aziz et al., 2024). This idea proposes a system that not only detects when multimodal NLP models make out-of-distribution or low-confidence predictions but also explains why the model might be unreliable in those cases. For example, if a clinical decision support system (CDSS) produces an anomalous recommendation for a rare patient profile, or if an automated essay grader is confused by a novel writing style, this tool would surface those cases and offer interpretable diagnostics—potentially using rationalization (Gurrapu et al., 2023) or heatmaps—giving practitioners a chance to intervene. This proactive, failure-centric XAI is largely absent from current literature and would significantly improve the trustworthiness and safety of NLP in high-stakes settings.

References:

1. Explainable Multimodal Sentiment Analysis Using Hierarchical Attention-Based Adaptive Transformer Models. Anna Shalini, B. M. Rao, Ranjitha. P. K, Guru Basava Aradhya S, S. Farhad, Elangovan Muniyandy, Y. El-Ebiary (2025). International Journal of Advanced Computer Science and Applications.
2. Rationalization for explainable NLP: a survey. Sai Gurrapu, Ajay Kulkarni, Lifu Huang, Ismini Lourentzou, Laura J. Freeman, Feras A. Batarseh (2023). Frontiers Artif. Intell..
3. Explainable AI in Healthcare: Systematic Review of Clinical Decision Support Systems. Noor A. Aziz, Awais Manzoor, Muhammad Deedahwar, M. Qureshi, Wael Rashwan (2024). medRxiv.