Unified Concept Incongruence Detection in Multimodal Prompts via Concept Schema Alignment and Mechanistic Prompt Probing

This research proposes a unified pipeline for detecting concept incongruence not just in text prompts, but jointly across multimodal inputs (e.g., text, images, audio). The idea is to extract and align underlying conceptual schemas from each modality, then algorithmically surface and explain when these schemas clash or violate common-sense/ontological constraints. It builds on essential concept extraction methods (e.g., AECD), cross-modality schema alignment using ontologies like ConceptNet or LLM world knowledge, and mechanistic prompt probing techniques such as attention heatmaps and concept activation probes to interpret model internal representations. The system aims to generate natural language rationales or clarifying questions for ambiguous cases and will be evaluated qualitatively and quantitatively on synthetic and real-world multimodal prompts with engineered and naturally occurring incongruence. This approach unifies prompt-based and mechanistic interpretability methods, generalizes concept incongruence detection beyond text to multiple modalities, and can improve user trust, safety, and alignment in AI systems across applications like creative generation and autonomous robotics.