Grid-Based Hierarchical Memory for Real-Time Multi-Modal Anomaly Detection

TL;DR: Let’s use a grid-based hierarchical memory architecture to spot weird things in complex data—think surveillance videos, sensor networks, or even mixed text and images. A concrete experiment: Extend GridHTM to handle multi-modal (video + audio) data, evaluating anomaly detection on a challenging event dataset.

Research Question: Can grid-based hierarchical memory models be generalized for real-time, multi-modal anomaly detection, outperforming current deep learning approaches in robustness and explainability?

Hypothesis: Such models will be more resilient to noise, concept drift, and data heterogeneity, making them ideal for unsupervised anomaly detection in complex, multi-source environments.

Experiment Plan: - Extend GridHTM to process and fuse multi-modal inputs (e.g., video frames and corresponding audio).

• Train and evaluate on datasets with labeled or semi-labeled anomalies (e.g., AVE, VIRAT, or custom sensor datasets).
• Compare performance, false positive/negative rates, and adaptability to concept drift with GANs, autoencoders, and standard HTM.
• Analyze internal memory activations to assess explainability and localization of detected anomalies.

References:

• Monakhov, V., Thambawita, V., Halvorsen, P., & Riegler, M. (2023). GridHTM: Grid-Based Hierarchical Temporal Memory for Anomaly Detection in Videos. Italian National Conference on Sensors.