Explainable AI for Data Quality Anomaly Diagnosis in Big Data Ecosystems

While Elouataoui et al. (2023, IEEE Access; Colloquium in Information Science and Technology) focus on holistic and intelligent anomaly detection frameworks for big data quality, their approaches—like many others—primarily emphasize detection and scoring, not interpretation or user guidance. Building on these foundations and the growing call for transparency (see Gao et al., 2024, Applied Energy), this idea proposes an XAI-powered anomaly detection pipeline that couples high-accuracy detection with interpretable explanations. For each identified anomaly (whether outlier, inconsistency, or missing value), the system would generate a natural-language rationale (e.g., “This value is anomalous because it violates expected temporal consistency, likely due to sensor drift”) and suggest targeted remediation (e.g., imputation, verification, or exclusion). This bridges the technical-user gap, enhances trust, and accelerates root-cause analysis—crucial for both domain experts and data stewards. Impact-wise, it could revolutionize data quality operations in regulated industries (healthcare, finance) where explainability is paramount.

References:

1. Quality Anomaly Detection Using Predictive Techniques: An Extensive Big Data Quality Framework for Reliable Data Analysis. Elouataoui Widad, Elmendili Saida, Youssef Gahi (2023). IEEE Access.
2. Big Data Quality Anomaly Scoring Framework Using Artificial Intelligence. Widad Elouataoui, Saida El Mendili, Youssef Gahi (2023). Colloquium in Information Science and Technology.
3. Enhancing anomaly detection accuracy and interpretability in low-quality and class imbalanced data: A comprehensive approach. Bixuan Gao, Xiangyu Kong, Shangze Li, Yi Chen, Xiyuan Zhang, Ziyu Liu, Weijia Lv (2024). Applied Energy.