Layerwise Causal Tracing: Visualizing the Propagation and Impact of Massive Activations Across Transformer Depth

TL;DR: Let’s create a “heatmap movie” showing how outlier activations and attention sinks move and interact through layers in real time. By tracing their journey, we’ll uncover new causal relationships and possibly spot intervention points for architecture tweaks.

Research Question: How do massive activations and attention sinks propagate and interact across layers in deep Transformer models, and what causal dependencies can be visualized?

Hypothesis: Visualizing the evolution of outliers and attention sinks throughout the network will reveal critical bottlenecks or “relay points” where interventions (e.g., gating, normalization) would be most effective at decoupling or controlling these phenomena.

Experiment Plan: Instrument a Transformer to log per-token, per-head activation norms and attention weights at every layer. Develop a visualization tool that animates the flow and concentration of outlier activations and sinks across the model’s depth. Correlate these visualizations with performance metrics and error cases. Introduce controlled architectural tweaks (e.g., gating, masking, alternative attention) at suspected “relay points” and observe effects on the propagation patterns and task performance.

References:

• Sun, S., Canziani, A., LeCun, Y., & Zhu, J. (2026). The Spike, the Sparse and the Sink: Anatomy of Massive Activations and Attention Sinks.
• Jilan, T. Z., Bhuiyan, M. H., Haldar, S., Chowdhury, M., & Bushra, N. (2025). An Interpretable Diagnosis of Retinal Diseases Using Vision Transformer and Grad-CAM. European Conference on Cognitive Ergonomics.